{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <center>手写数字分类识别入门体验</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 实现一个图片分类应用\n",
    "## 概述\n",
    "下面我们通过一个实际样例，带领大家体验MindSpore基础的功能，对于一般的用户而言，完成整个样例实践会持续20~30分钟。\n",
    "\n",
    "本例子会实现一个简单的图片分类的功能，整体流程如下：\n",
    "\n",
    "1. 处理需要的数据集，这里使用了MNIST数据集。\n",
    "\n",
    "2. 定义一个网络，这里我们使用LeNet网络。\n",
    "\n",
    "3. 自定义回调函数收集模型的损失值和精度值。\n",
    "\n",
    "4. 加载数据集并进行训练，训练完成后，查看结果及保存模型文件。\n",
    "\n",
    "5. 加载保存的模型，进行推理。\n",
    "\n",
    "6. 验证模型，加载测试数据集和训练后的模型，验证结果精度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "说明：<br/>你可以在这里找到完整可运行的样例代码：<https://gitee.com/mindspore/docs/blob/master/tutorials/tutorial_code/lenet/lenet.py>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 训练的数据集下载"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 方法一：\n",
    "从以下网址下载，并将数据包解压缩后放至Jupyter的工作目录下：<br/>训练数据集：{\"<http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz>\"}\n",
    "<br/>测试数据集：{\"<http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz>\", \"<http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz>\"}<br/>我们用下面代码查询jupyter的工作目录。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:44.914340Z",
     "start_time": "2020-10-20T07:21:44.905756Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\Administrator'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "\n",
    "os.getcwd()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练数据集放在----`Jupyter工作目录+\\datasets\\MNIST_Data\\train\\`，此时train文件夹内应该包含两个文件，`train-images-idx3-ubyte`和`train-labels-idx1-ubyte` <br/>测试数据集放在----`Jupyter工作目录+\\datasets\\MNIST_Data\\test\\`，此时test文件夹内应该包含两个文件，`t10k-images-idx3-ubyte`和`t10k-labels-idx1-ubyte`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 方法二：\n",
    "直接执行下面代码，会自动进行训练集的下载与解压，但是整个过程根据网络好坏情况会需要花费几分钟时间。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:44.969944Z",
     "start_time": "2020-10-20T07:21:44.915348Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "******Downloading the MNIST dataset******\n"
     ]
    }
   ],
   "source": [
    "import urllib.request   \n",
    "from urllib.parse import urlparse\n",
    "import gzip \n",
    "import os\n",
    "\n",
    "def unzip_file(gzip_path):\n",
    "    \"\"\"\n",
    "    unzip dataset file\n",
    "    \n",
    "    Args:\n",
    "        gzip_path (str): dataset file path\n",
    "    \"\"\"\n",
    "    open_file = open(gzip_path.replace('.gz',''), 'wb')\n",
    "    gz_file = gzip.GzipFile(gzip_path)\n",
    "    open_file.write(gz_file.read())\n",
    "    gz_file.close()\n",
    "    \n",
    "def download_dataset():\n",
    "    \"\"\"Download the dataset from http://yann.lecun.com/exdb/mnist/.\"\"\"\n",
    "    print(\"******Downloading the MNIST dataset******\")\n",
    "    train_path = \"./datasets/MNIST_Data/train/\" \n",
    "    test_path = \"./datasets/MNIST_Data/test/\"\n",
    "    train_path_check = os.path.exists(train_path)\n",
    "    test_path_check = os.path.exists(test_path)\n",
    "    if train_path_check == False and test_path_check == False:\n",
    "        os.makedirs(train_path)\n",
    "        os.makedirs(test_path)\n",
    "    train_url = {\"http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\"}\n",
    "    test_url = {\"http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\", \"http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\"}\n",
    "    \n",
    "    for url in train_url:\n",
    "        url_parse = urlparse(url)\n",
    "        # split the file name from url\n",
    "        file_name = os.path.join(train_path,url_parse.path.split('/')[-1])\n",
    "        if not os.path.exists(file_name.replace('.gz', '')):\n",
    "            file = urllib.request.urlretrieve(url, file_name)\n",
    "            unzip_file(file_name)\n",
    "            os.remove(file_name)\n",
    "            \n",
    "    for url in test_url:\n",
    "        url_parse = urlparse(url)\n",
    "        # split the file name from url\n",
    "        file_name = os.path.join(test_path,url_parse.path.split('/')[-1])\n",
    "        if not os.path.exists(file_name.replace('.gz', '')):\n",
    "            file = urllib.request.urlretrieve(url, file_name)\n",
    "            unzip_file(file_name)\n",
    "            os.remove(file_name)\n",
    "\n",
    "download_dataset()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这样就完成了数据集的下载解压缩工作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 处理MNIST数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于我们后面会采用LeNet这样的卷积神经网络对数据集进行训练，而采用LeNet在训练数据时，对数据格式是有所要求的，所以接下来的工作需要我们先查看数据集内的数据是什么样的，这样才能构造一个针对性的数据转换函数，将数据集数据转换成符合训练要求的数据形式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 查看原始数据集数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:46.072389Z",
     "start_time": "2020-10-20T07:21:44.970964Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The type of mnist_ds: <class 'mindspore.dataset.engine.datasets.MnistDataset'>\n",
      "Number of pictures contained in the mnist_ds： 60000\n",
      "The item of mnist_ds: dict_keys(['image', 'label'])\n",
      "Tensor of image in item: (28, 28, 1)\n",
      "The label of item: 3\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAEICAYAAACZA4KlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAANkUlEQVR4nO3de6xl5V3G8e8jHYYINEIRmAIWRExEU4d6pE1Qg0FbSo3QP0qYmIYmtYMWEkkwKUGTElMiMS0VU3sZBDvUltJYCJigBdGEoIZwIJRLQUAyLdMZmTZQC7QMA/z846zRw+HcZu+1L2fe7yfZ2Wuv629W5jlr7fWutd9UFZL2fz8x6QIkjYdhlxph2KVGGHapEYZdaoRhlxph2PUGSbYl+a1J16F+GXaNXZK/S7IzyQ+TPJ7k9yddUwsMu0YmyZuWmPTnwPFV9Wbgd4FPJPmV8VXWJsO+hnSn13+c5MEk/5PkxiQHJflQkrsXzFtJfq4b/mKSzyb5xyQvJPm3JEcn+cskzyV5LMkpCzb3q0m+1U3/2yQHzVv37yR5IMkPkvx7krcvqPFjSR4EXlws8FX1SFXt3vuxe53Y027SEgz72nMucCZwAvB24EP7sNyfAkcAu4H/AO7vPv89cNWC+X8PeA9zIfz5blmSvAO4DrgAeAvwBeDWJOvnLbsJeB/wU1X1SveH5rPzV96N+xHwGLATuG2V/w4NyLCvPX9VVTuq6lngH4CNq1zu5qq6r6peAm4GXqqq66vqVeBGYOGR/TNV9XS3nSuYCzDAR4AvVNU9VfVqVW1l7o/HuxbU+HRV/Rigqj5aVR+dv/Lu86HArwM3devQCBn2tee/5w3/CDhklcs9M2/4x4t8Xriep+cNfxt4azf8NuCS7hT+B0l+ABw3b/rCZZfU/bG4GzgW+MPVLKPBLXUBRWvLi8BP7v2Q5Oge1nncvOGfAXZ0w08DV1TVFcssu6+PUr4Jv7OPnEf2/cM3gV9MsrG7kHZ5D+u8MMmxSQ4HLmPuVB/gGuAPkrwzcw5O8r4kh65mpUmOTHJekkOSHJDkPcx9RfiXHmrWMgz7fqCqHgf+DPhn4Ang7uWXWJWvALcDT3WvT3TbmmXue/tngOeAJ1nhImGSzyf5/N5ymTtl394t/0ng4qq6pYeatYz44xVSGzyyS40w7FIjDLvUCMMuNWKs7ewHZn0dxMHj3KTUlJd4kZdrdxabNlTYk5wJXA0cAPxNVV253PwHcTDvzBnDbFLSMu6pO5ecNvBpfJIDgL8G3gucDGxKcvKg65M0WsN8Zz8VeLKqnqqql4GvAmf3U5akvg0T9mN4/QMP27txr5Nkc5LZJLN7fLBJmphhwr7YRYA33I5XVVuqaqaqZtaxfpFFJI3DMGHfzuufjDqW/38yStKUGSbs9wInJTkhyYHAecCt/ZQlqW8DN711Pzd0EfAN5prerquqR3qrTFKvhmpnr6rb8LfDpDXB22WlRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRgzVi6v0jR0PTLqEkXjPWzdOuoTeDRX2JNuA54FXgVeqaqaPoiT1r48j+29W1fd7WI+kEfI7u9SIYcNewO1J7kuyebEZkmxOMptkdg+7h9ycpEENexp/WlXtSHIkcEeSx6rqrvkzVNUWYAvAm3N4Dbk9SQMa6sheVTu6913AzcCpfRQlqX8Dhz3JwUkO3TsMvBt4uK/CJPVrmNP4o4Cbk+xdz1eq6p96qUqvs7+2ZWu8Bg57VT0F/HKPtUgaIZvepEYYdqkRhl1qhGGXGmHYpUb4iOsUsGlN4+CRXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRtjOribtjz8VvRKP7FIjDLvUCMMuNcKwS40w7FIjDLvUCMMuNcJ29ikwyjbflZ6VH3bbPou/dnhklxph2KVGGHapEYZdaoRhlxph2KVGGHapEbaz7+dWakffn9vJW3xmfTkrHtmTXJdkV5KH5407PMkdSZ7o3g8bbZmShrWa0/gvAmcuGHcpcGdVnQTc2X2WNMVWDHtV3QU8u2D02cDWbngrcE7PdUnq2aAX6I6qqp0A3fuRS82YZHOS2SSze9g94OYkDWvkV+OraktVzVTVzDrWj3pzkpYwaNifSbIBoHvf1V9JkkZh0LDfCpzfDZ8P3NJPOZJGZcV29iQ3AKcDRyTZDnwcuBL4WpIPA98BPjDKIqXF2I6+b1YMe1VtWmLSGT3XImmEvF1WaoRhlxph2KVGGHapEYZdaoSPuK4B+/NjqMuxaa1fHtmlRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqE7exj0Go7+UpsRx8vj+xSIwy71AjDLjXCsEuNMOxSIwy71AjDLjXCdnZNzEr3H9gO3y+P7FIjDLvUCMMuNcKwS40w7FIjDLvUCMMuNcJ2dk0t2+H7teKRPcl1SXYleXjeuMuTfDfJA93rrNGWKWlYqzmN/yJw5iLjP11VG7vXbf2WJalvK4a9qu4Cnh1DLZJGaJgLdBclebA7zT9sqZmSbE4ym2R2D7uH2JykYQwa9s8BJwIbgZ3Ap5aasaq2VNVMVc2sY/2Am5M0rIHCXlXPVNWrVfUacA1war9lSerbQGFPsmHex/cDDy81r6TpsGI7e5IbgNOBI5JsBz4OnJ5kI1DANuCCEdaoIQzbFu1v3u8/Vgx7VW1aZPS1I6hF0gh5u6zUCMMuNcKwS40w7FIjDLvUCB9xHQMfxRyM+61fHtmlRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEz7M3zp+KbodHdqkRhl1qhGGXGmHYpUYYdqkRhl1qhGGXGrGaLpuPA64HjgZeA7ZU1dVJDgduBI5nrtvmc6vqudGVOlmTbI8e5vfTp7kd3d+FH6/VHNlfAS6pql8A3gVcmORk4FLgzqo6Cbiz+yxpSq0Y9qraWVX3d8PPA48CxwBnA1u72bYC54yqSEnD26fv7EmOB04B7gGOqqqdMPcHATiy7+Ik9WfVYU9yCPB14OKq+uE+LLc5yWyS2T3sHqRGST1YVdiTrGMu6F+uqpu60c8k2dBN3wDsWmzZqtpSVTNVNbOO9X3ULGkAK4Y9SYBrgUer6qp5k24Fzu+Gzwdu6b88SX1ZzSOupwEfBB5Ksrcd5zLgSuBrST4MfAf4wGhK1DQ3n2ntWDHsVXU3kCUmn9FvOZJGxTvopEYYdqkRhl1qhGGXGmHYpUYYdqkR/pS0RsrHWKeHR3apEYZdaoRhlxph2KVGGHapEYZdaoRhlxphO7uGYjv62uGRXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRtjOvkqjbE9eq91Ba23xyC41wrBLjTDsUiMMu9QIwy41wrBLjTDsUiNWbGdPchxwPXA08BqwpaquTnI58BHge92sl1XVbaMqdH9mW7fGYTU31bwCXFJV9yc5FLgvyR3dtE9X1SdHV56kvqwY9qraCezshp9P8ihwzKgLk9SvffrOnuR44BTgnm7URUkeTHJdksOWWGZzktkks3vYPVSxkga36rAnOQT4OnBxVf0Q+BxwIrCRuSP/pxZbrqq2VNVMVc2sY30PJUsaxKrCnmQdc0H/clXdBFBVz1TVq1X1GnANcOroypQ0rBXDniTAtcCjVXXVvPEb5s32fuDh/suT1JfVXI0/Dfgg8FCSvc9iXgZsSrIRKGAbcMFIKpTUi9Vcjb8byCKTbFOX1hDvoJMaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRhh2qRGGXWqEYZcaYdilRqSqxrex5HvAt+eNOgL4/tgK2DfTWtu01gXWNqg+a3tbVf30YhPGGvY3bDyZraqZiRWwjGmtbVrrAmsb1Lhq8zReaoRhlxox6bBvmfD2lzOttU1rXWBtgxpLbRP9zi5pfCZ9ZJc0JoZdasREwp7kzCT/meTJJJdOooalJNmW5KEkDySZnXAt1yXZleTheeMOT3JHkie690X72JtQbZcn+W637x5IctaEajsuyb8meTTJI0n+qBs/0X23TF1j2W9j/86e5ADgceC3ge3AvcCmqvrWWAtZQpJtwExVTfwGjCS/AbwAXF9Vv9SN+wvg2aq6svtDeVhVfWxKarsceGHS3Xh3vRVtmN/NOHAO8CEmuO+WqetcxrDfJnFkPxV4sqqeqqqXga8CZ0+gjqlXVXcBzy4YfTawtRveytx/lrFborapUFU7q+r+bvh5YG834xPdd8vUNRaTCPsxwNPzPm9nuvp7L+D2JPcl2TzpYhZxVFXthLn/PMCRE65noRW78R6nBd2MT82+G6T782FNIuyLdSU1Te1/p1XVO4D3Ahd2p6tanVV14z0ui3QzPhUG7f58WJMI+3bguHmfjwV2TKCORVXVju59F3Az09cV9TN7e9Dt3ndNuJ7/M03deC/WzThTsO8m2f35JMJ+L3BSkhOSHAicB9w6gTreIMnB3YUTkhwMvJvp64r6VuD8bvh84JYJ1vI609KN91LdjDPhfTfx7s+rauwv4Czmrsj/F/Ank6hhibp+Fvhm93pk0rUBNzB3WreHuTOiDwNvAe4EnujeD5+i2r4EPAQ8yFywNkyotl9j7qvhg8AD3eusSe+7Zeoay37zdlmpEd5BJzXCsEuNMOxSIwy71AjDLjXCsEuNMOxSI/4XNXZAvG/0zb4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from mindspore import context\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "import numpy as np\n",
    "import mindspore.dataset as ds\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE, device_target=\"CPU\") \n",
    "train_data_path = \"./datasets/MNIST_Data/train\"\n",
    "test_data_path = \"./datasets/MNIST_Data/test\"\n",
    "mnist_ds = ds.MnistDataset(train_data_path)\n",
    "print('The type of mnist_ds:', type(mnist_ds))\n",
    "print(\"Number of pictures contained in the mnist_ds：\", mnist_ds.get_dataset_size())\n",
    "\n",
    "dic_ds = mnist_ds.create_dict_iterator()\n",
    "item = dic_ds.get_next()\n",
    "img = item[\"image\"].asnumpy()\n",
    "label = item[\"label\"].asnumpy()\n",
    "\n",
    "print(\"The item of mnist_ds:\", item.keys())\n",
    "print(\"Tensor of image in item:\", img.shape) \n",
    "print(\"The label of item:\", label)\n",
    "\n",
    "plt.imshow(np.squeeze(img))\n",
    "plt.title(\"number:%s\"% item[\"label\"].asnumpy())\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面的运行情况我们可以看到，训练数据集`train-images-idx3-ubyte`和`train-labels-idx1-ubyte`对应的是6万张图片和6万个数字下标，载入数据后经过`create_dict_iterator`转换字典型的数据集，取其中的一个数据查看，这是一个key为`image`和`label`的字典，其中的`image`的张量(高度28，宽度28，通道1)和`label`为对应图片的数字。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 数据处理\n",
    "\n",
    "数据集对于训练非常重要，好的数据集可以有效提高训练精度和效率，在加载数据集前，我们通常会对数据集进行一些处理。\n",
    "\n",
    "#### 定义数据集及数据操作\n",
    "\n",
    "我们定义一个函数`create_dataset`来创建数据集。在这个函数中，我们定义好需要进行的数据增强和处理操作：\n",
    "\n",
    "1. 定义数据集。\n",
    "2. 定义进行数据增强和处理所需要的一些参数。\n",
    "3. 根据参数，生成对应的数据增强操作。\n",
    "4. 使用`map`映射函数，将数据操作应用到数据集。\n",
    "5. 对生成的数据集进行处理。\n",
    "\n",
    "定义完成后，使用`create_datasets`对原始数据进行增强操作，并抽取一个`batch`的数据，查看数据增强后的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:46.091026Z",
     "start_time": "2020-10-20T07:21:46.073390Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of groups in the dataset: 1875\n"
     ]
    }
   ],
   "source": [
    "import mindspore.dataset.vision.c_transforms as CV\n",
    "import mindspore.dataset.transforms.c_transforms as C\n",
    "from mindspore.dataset.vision import Inter\n",
    "from mindspore.common import dtype as mstype\n",
    "\n",
    "\n",
    "def create_dataset(data_path, batch_size=32, repeat_size=1,\n",
    "                   num_parallel_workers=1):\n",
    "    \"\"\" \n",
    "    create dataset for train or test\n",
    "    \n",
    "    Args:\n",
    "        data_path (str): Data path\n",
    "        batch_size (int): The number of data records in each group\n",
    "        repeat_size (int): The number of replicated data records\n",
    "        num_parallel_workers (int): The number of parallel workers\n",
    "    \"\"\"\n",
    "    # define dataset\n",
    "    mnist_ds = ds.MnistDataset(data_path)\n",
    "\n",
    "    # define some parameters needed for data enhancement and rough justification\n",
    "    resize_height, resize_width = 32, 32\n",
    "    rescale = 1.0 / 255.0\n",
    "    shift = 0.0\n",
    "    rescale_nml = 1 / 0.3081\n",
    "    shift_nml = -1 * 0.1307 / 0.3081\n",
    "\n",
    "    # according to the parameters, generate the corresponding data enhancement method\n",
    "    resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR)\n",
    "    rescale_nml_op = CV.Rescale(rescale_nml, shift_nml)\n",
    "    rescale_op = CV.Rescale(rescale, shift)\n",
    "    hwc2chw_op = CV.HWC2CHW()\n",
    "    type_cast_op = C.TypeCast(mstype.int32)\n",
    "\n",
    "    # using map to apply operations to a dataset\n",
    "    mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns=\"label\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=resize_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns=\"image\", num_parallel_workers=num_parallel_workers)\n",
    "    \n",
    "    # process the generated dataset\n",
    "    buffer_size = 10000\n",
    "    mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size)\n",
    "    mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True)\n",
    "    mnist_ds = mnist_ds.repeat(repeat_size)\n",
    "\n",
    "    return mnist_ds\n",
    "\n",
    "datas = create_dataset(train_data_path)\n",
    "print('Number of groups in the dataset:', datas.get_dataset_size())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上述增强过程中：\n",
    "- 数据集中的`label`数据增强操作：\n",
    "    - `C.TypeCast`：对数据类型转化为`int32`。\n",
    "- 数据集中的`image`数据增强操作：\n",
    "    - `datasets.MnistDataset`：将数据集转化为MindSpore可训练的数据。 \n",
    "    - `CV.Resize`：对图像数据像素进行缩放，适应LeNet5网络对数据的尺寸要求。\n",
    "    - `CV.Rescale`：对图像数据进行标准化，归一化操作，使得每个像素的数值大小在范围（0,1）中，可以提升训练效率。\n",
    "    - `CV.HWC2CHW`：对图像数据张量进行变换，张量形式由高x宽x通道（HWC）变为通道x高x宽（CHW），方便进行数据训练。\n",
    "- 其他增强操作：\n",
    "    - `mnist_ds.shuffle`：本例中表示随机将数据存放在可容纳10000张图片地址的内存中进行混洗。\n",
    "    - `mnist_ds.batch`：本例中表示从混洗的10000张图片地址中抽取32张图片组成一个`batch`。\n",
    "    - `mnist_ds.repeat`：将`batch`数据进行复制增强。\n",
    "\n",
    "> 先进行`shuffle`、`batch`操作，再进行`repeat`操作，这样能保证1个`epoch`内数据不重复。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "调用数据增强函数后，查看数据集`size`由60000变成了1875，符合我们的数据增强中`mnist_ds.batch`操作的预期（$60000/32=1875$）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 进一步查看增强后的数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从1875组数据中取出一组数据查看其数据张量及`label`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:46.521303Z",
     "start_time": "2020-10-20T07:21:46.092033Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor of image: (32, 1, 32, 32)\n",
      "labels: [0 1 2 1 2 7 5 4 7 1 0 0 6 4 4 1 3 0 3 2 2 7 8 5 4 8 2 2 1 5 4 8]\n"
     ]
    }
   ],
   "source": [
    "data = datas.create_dict_iterator(output_numpy=True).get_next()\n",
    "images = data[\"image\"]\n",
    "labels = data[\"label\"]\n",
    "print('Tensor of image:', images.shape)\n",
    "print('labels:', labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "将张量数据和下标对应的值进行可视化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:47.141782Z",
     "start_time": "2020-10-20T07:21:46.522315Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAADsCAYAAABKZHxbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyde3hU1bn/P2tmQi4kEJJwS7iEAAmjWAERGxRBPXgtVI+AWigV5cQGqR4R7QV/Sj1ytArSIqJGuRShVKRqxUvBYwkFRCgFUWogQAjhDgkEyZVkZv3+2DOTmWRuSWbPnuD6PE+ezN57zd7fWXuvd6/L+64lpJQoFAqFIvyYjBagUCgU31eUAVYoFAqDUAZYoVAoDEIZYIVCoTAIZYAVCoXCIJQBVigUCoNQBlihUCgMImINsBAiSQjxvhCiUghxWAjxE6M1+UIIMV0IsUMIUSuEWGa0Hl8IIaKFEIsd+XlBCLFLCHGb0bp80RbytQ3maUWjP5sQ4hWjdXlDCJEvhKhx07rPaE2BEEL0d2heEUx6i96CWsGrwEWgKzAI+FgIsVtK+W9jZXnlOPAccAsQa7AWf1iAI8BIoAS4HVgthLhCSllspDAftIV8bVN5KqWMd34WQrQHTgHvGqcoINOllG8ZLaIZvAr8M+jUUkq/f0AxMBP4GjgPvAPEAPcDmxullUA/x+dlwCLgU6AC2AJ0A34PnAP2AoN9XLM9mvHNdNv3NvBCpGltdM7ngGWB0kWCVrdzfw3cHclag81Xo3W2pTx1nOtnQBEgIlErkA9MDTbfjc5X4F5gNTAbWBGM3mC7ICYAtwJ9gB84fkyw33sKSAFqga3ATsf2GuBlZ0IhxCIhxCLHZiZgk1IWup1rN3B5BGptDYZqFUJ0RcvrYFoVbSVfVZ4Gn6c/A5ZLh/WIUK3PCyFKhRBbhBCjmnHNsGoVQnQAngUeD/JaQPB9wAuklMellGeBtWhdAsHwvpTyX1LKGuB9oEZKuVxKaUN7Mw12JpRSTpNSTnNsxqO9vdw5DyREoNbWYJhWIUQUsBL4o5RybyRrbSYqT4PT2gut2+SPQV7TCK2/BDKANCAPWCuE6BuhWv8HWCylPBLktYDgDfBJt89VaAYyGE65fa72su3rPBVAh0b7OgAXgrhmuLW2BkO0CiFMaF06F4HpQV6zreSrytPgzjMZrUl+KMhrhl2rlHKblPKClLJWSvlHtG6B2yNNqxBiEPAfwPwgr+OiNYNwlUCcm4hurThXYwoBixCiv5Ryv2PflQTXrPOGnlpDja5ahRACWIw2uHm7lLKuFadrK/mq8rQpk4EXWnmOcN9/CYgWfldPraOAdKBEexSIB8xCiMuklEP8fbE1bmi7gcuFEIOEEDFoHc8hQUpZCbwHPCuEaC+EuBb4MVoNoyXophVACGFxnNeMlvExQoiWvtx01Qq8BliBMVLK6laeq63kq8pTN4QQw9Ga9a31ftBNqxAiUQhxi/OeCyEmAtcD6yJNK1r3SF+0ro5BwOvAx2jeO35psQF2DJA9C/wfsB/Y3NJzAQghXhdCvO62axqa69FpYBWQK1voghYGrU+hNVF+BUxyfH4q0rQKIXoDD6E9JCfd/CsnRppWByHJV5WnHnkK2uDbe1LKYLr0fKKz1ig075czQCnwC+BOKWWLfIH11CqlrJJSnnT+oXWh1kgpzwQ8T3ADoAqFQqEINREbCadQKBSXOsoAKxQKhUEoA6xQKBQGoQywQqFQGIQywAqFQmEQQflUjjaNjyhXic/s73p1xm4rOkFpbQ2Xgta2ohOU1tbgTyuoGrBCoVAYhjLACoVCYRCRPCG7QhFW5LWD2P+A9yKRsiUKgKQlW8MpSXGJEzYDbIqLo+TRQdijG/Z1/qoegNgPtodLhsIgGt//tPwazPk7jRXlwDZKmy/lSG49h0Z4X3yhD1MBSFoSNlmK7wGqC0KhUCgMIiw1YHPnzpyY0J91uS/Sw9IwpWafT7VaReYH4VDhHdOVVgBKhyQCYK6TJL67C1lba5yoSxCREM+ah+ZibafNCGiV0+iVb6wmJ8dGxgCwb4S2wEGFvYZxhXezoO9qMqPaGyntkkVeO4hzmf6X+UvZWY59d0GYFGmYO3fmzI/6uba7bDxBfVGxbtcLiwGuu6wHu2YtQp85zVtH8Z2dACh4SCt8h+oquP/8DOLyC7BfaNVkUYo2QkyZ9v+50gEAHK5OxjS+mtUbruKplGAWtlAEQlgs2K8ZiLRoXlmnH6th97Blfr/T58McrPMyALDtL9JbIqDZqn/Oec21Pew3uXRqywZYREdzsUOU3pcJGX2i4tn4Rh6j75uCaeMuo+UowkCXhV8AsGmhVhNGVGFOiSPa1Jp51VuGiI7GlNjRY5+9/HybaZGZEztCdHTTA8mJLPjTq81qURwam0efKEcr+UH9DXBjW3WorgJTvb5uxbob4PLxg1k1Zy6RWPtVKLxhTkkh54ut3BZ3Dm1a2vBRPn4wS5972WPflKdm0HHFl2HV0VKOLUvlz4MWN9lvFpK+Fv9dDkbT2FbdP30Giet3oacJ1n0Qzm4R9InybnzfufE13rnxNQrfGqq3DJ9kLCkhY0kJ1jzPdQvHv76Oqv+8xiBV/qn6z2sY8XWN608MHWi0pGZhzZtGxpISo2X4xiTIijpNtNCMb58Pc7C+cBbrC2d1v7TdIrC2i/P4m/H0Kk5PH677tUNBp7jqJvqt7eLIjGqPWUT2mH9jW9XuuzpXy+P09OGcnj6cEV/XcPajzJBdU7cacPnkbABG/GKbzzTDorUHvHevUr1kBKT+yFEAMpZbsDKNghytLzin43He7GhqWEQqzJgTO3JsWSqd4pqucDMs+Z8efZOTFnTjaEW61/Ocq4ol7f7j2MobLzJtHPGHpSvfIw3TwAHYF1ygp0UzFhnvP8SAvO9074P0V14mxJ/n8M/XsTxGW+Emde4XumppDWJeCqOfHMNn1rUB044uGANA+YoePtP0L2ztCk+BCWSrjs8czuT7tZWQnkg6CMAmYkJybV0M8NkHshmYsweAed09fT37fJjDtOs+d/0QgP/qvYln50wgfZZxTu71RcX02NAJcgyTADQEA5iibey46lU6mQO/Alak5/s8ds5WxdCFD2OvNQPQc61J+V37oT45ls8G/BkcBSx5hyksI/HfZWiDU43Li5Mnkg5S9dN8AJZZtdqwqDGT9atv/Q4WmxIS2PfCZcgYG6D//W+3bgfVscO4PvcuAP5xxfsex/t8mEPMSc3sOOMAkj4wNrilcd6ftlVy3cqZ9C86Qj1QYb3oYa9CiS4GuPTaOpb22uSx75ytiqEbH8Y6r5SPM67w+EETE8o4NGYDm2aF5q3SljmXGcuh25yjsJrxHV0whgPFXZt9rg7JlewetoqDNy517bu+x11UMwxQATBtjWc6f6v9v037f9pWyXUXZmKu8T3fiy1G8uXYuXQxa4Nf1iPT6KWj22f55Gy+yxB09nG836o6TBsj57mrvnMYnbNPeOwrswn6zT9I/anTlE/OZtTle1zHVl5I5u21N5BOaF4akd0po1AoFJcwIa8BX7xlKP3STzXZXyntdNgaC+VHAeXcHghnMEBNfRRiXgqZ63Y0+xymgQMY9fKdAORlrSQzqj3/uOJ9+owxPgAmErFkpFN0Q8NI/aTiUcQfD5MrWguG2ruY21M4+bXACd3Lm86TNcZOOsG2gW3nwToyxs4ht26SwrpK7v1qKmm1xwHt97i35t88PCKkXaUhN8Dy8VKvHfA9LPHsmrWI0XumAOdCfVldEAbMLBpz3s5zpQNcwQDRZceA4hady75nL9E3a58nfjSFlVcsjYjILiPyNRjOXN/dNQgLcOrXfWi3sfkvPoVvyqwxdPu2M7YzAVdsN4T8qv50mR9D3ZVaAEiP+EMAbK/VXsSHS1LIbGF59EZYIuFqZR1FddoPEPWSc1WxnLNVBTXA9H0j7r1tbHovBqh0/AXGnJwEFv+3svOD5TyyagJ/G/Bx60U2E2GxQHIi5gi1vKY47Tmsa6/1pdqknYP11QidnfD94dRgkw39u8lmTY+zP7ctsvPp1xhWk0unZZFpgHM6Hidn1dIm++/5ey4AmVND+0IOiwFecG4AG27qC4CpbA9p97dn6MKHPQaHFC0n7ZNaZnT9KGA6zbUq/AOd9msGsuBPr0asI37Jo4MAWJf7IhDPwfpq/vu2KZgK9+jdYveJUwNl5a59Bx7TylBw3Q6KtoDuBnhS8ShKH+mBPNUwkmgrP+9yi1IEpvCtoTw4bLPP4zmdtkR0rUhaRER0ffjC3k7775woqrelHTes/he19ijWzR5J3Hu+fdn1wiYFlJVjO3Xata//q5rQ0Z9O8Ug7/vV15HQ8HvCczgCY+tBKbTYzf/MnCmd0d20v3n5dyGuWwWJ94Syjl0/xesw9X9+5UXvp3fNWbki1htQAF741lKV9PSdMPVqRSPSOPR77il7I5rnrVofy0rog/a7mpC/ugRhL+y5hVKzdT+rgjVvWpsn0X2JsETQyXxtzfOZwHhi/zmNftIhyuUmemxXH5vhsEpcbPxG7M3jF1CiI5e1ZY3izo4nESUd9BkBY86aRsfx4RATATIg/D/ENgUH/12uAYVps+4sw7ffc5yx7o+L2A+3J2jSZDn/Tylh6iAdlQ2qAHxy2OYCh0EgbcoKJCWWhvPSlR3Q0fx602DV9I8CUkhEAfPm3K5okf2DcOr/O4k4H+J75NYgtxk2E3ufDHAbsLCfwUxIeAjnZz+u+kynT2rPHokVL6bYiRiteSnHvbaN85nCuS/H9O3psqNZ1WkWA4jnZPN17Ndd/cxdntnb3muYPk97k5rjwT3LULBxlz9lqMxXEk7RUn+hD3bogfnvmMgDK1qeSSrHHiggPpK4L8O3vN5aePdj/cE/XoIszEMO5LE6vJZ4Pg695ArbX1rkGD6zzSsM2pZ87piutFN7XMKFNv1V1YZ/j1ReNnex9sbTXJvpcmwXotyKGMyrMOUe2qDGTVfVtUN89PV0LlfX2InEGQGUdP48tdHK98tMxG5iYUMZzW7vTa7Z3g/WLqKk8PW61RwXsjtRv+NP0W1yz0hmJt7LnvDd6oAIxFAqFwiB0qwEv26HVyjIdE4c0XhHBnfxqk9YRj7E+l5ayam7dewdrMv9CvMm4sOiLGZ0dI91aE+jg3lRSdjS8K89OyfZI/+tHVmr9amihkqA5jB8uSXENGOhd+/FF6ZBEDo2NzFH7xk72oDni5+yb6Nqe3ffDoLrVWq3FERbuHhwT7FXvzvm719pvYV0l43dNJWt6UXgnY/LjOpI+aytvXj2CiW7BGk8kHaQ25+8N8zEbiHvZm1Q8ivoXuxK7Tr/Qad0McIdkzYfVPnIwADUdoogRno+U07l5yobQjiy2FPuevZjv68KRbXas7YwLGDBX1vHS2b480mkv0SKKorvegLv8f2d1RUcKa7rz9tobAO1BD6XDeEuJUNdfn6w+fxXRNxe7tmevH0v+wA9cz7Np4AA4fIz6IQ3L1kR9e9SQwALnKhNdotZ7Pb76/FV0u7NA95dvIB3uaaRFuIIbIg1z586csTa8BMIRiKObAd49bJX2YZX7Xs95gfVybm7ryB172HBDBv2/OMVtcedc89I6qbDXcKTe82X28rM/p+OKL0M2SUhricTgBnfMyUnEWDwHgyrsNRyuTsZbAIzzeb51wR1Uz7ey8Y0817FhvzEmsMCUnMT8lYu8tir9/ZZw6HDef5HgKPN+VsQIp1Z/lN7Rj51Ph7e1FrZl6RXNw1ZaSt7wbPZvKGrSvBxXeDem+zyNR2K5vjP3N5dIDG5wJ+2TWuanfYJ7YMq4wrsxja/GnyFYk/kXziysJ9JXeAnmt+iJ8/6veWgu4H9FDKO1GokhBtiaN40eG6qxHtdWGDCqfzKikRLbmTN8fn8269tf73HIUlaN7VRkLxbZOLjBGVgg640OA9DoHVvm6ue/cvt9AKQ+JbGVHfNIFzsjhiufu89VA443xRDvNnQ95Nlcun18IOzPsBg6kBuWbSUjyrN15O+3hIs//PRN12dvtfPG1NRHOeY8+f4RUgO8bvZIzs2K8zmpNLg5hBcVR7zhjYSAAbljTxNXlUjxoW3LrJs9kr921PrLuzpWXbDv+apJOvuevXR9aRBXZ+Z6PU+3jw4Y0v9ra+8MFvE0wN+VaU38bnuM69YL1s/XuSKGmJdCSyecCiVGlPeQGuC497axOT4ba8YPfabJWFwSEdE4Cn1J21gDgFVoa+2ZaqHXhaYGziji3tsW9HJTYstXJG3xfsyISoS8dhCnH6tpsv/6b+6i59rI9iz1tiKGnl4GLaHxihh6EvIuiMTlW0n0czwyGqC+kVXV3PHxY8gYW1jWo7pUMedrraBe+Q37VM09NJzLjGX3sGUe+0YXjIHXOhuyyol7mQmEUQFBwdCpsJo+n07VgmDm7KHez1JPoSKyX5cKhUJxCaO8IBphv3CB/g+Hf/YrhSJY4o/XMal4lMdirOUrehi2uGVzykwkj/uILV+R6ehqCldrTRlghaKN0W7dDkrLBvLckoZZxGLOqw6etogywApFG0Tu2MOmHzT4MMehWm1tESFlJLjFKxQKxfcPNQinUCgUBqEMsEKhUBiEMsAKhUJhEMoAKxQKhUEoA6xQKBQGEbEGWAhR0ejPJoR4xWhd3hBCTBdC7BBC1Aohlhmtxx9CiCQhxPtCiEohxGEhxE+M1uSLNqb1XiFEgUPrQSHECKM1+UMI0V8IUSOEWGG0lkC0Ba0ttQER6wcspXRNuCqEaA+cAt41TpFfjgPPAbcA3ic9jRxeBS4CXYFBwMdCiN1Syn8bK8srbUKrEGI08DvgHmA74H1J4MjiVeCfRosIkragtWU2QErp9w9tnriZwNfAeeAdtFms7wc2N0orgX6Oz8uARcCnQAWwBegG/B44B+wFBge6vuNcPwOKcPgtR6pWxw1YFuRvCrtWtEXmLgKZbvveBl64FLQadf+BL4AHg7nvRmt1nONeYDUwG1ihtBpjA6SUQXdBTABuBfoAP3D8mGC/9xSQAtQCW4Gdju01wMvOhEKIRUKIRT7O8zNguXT8wgjX2hzCrTUTsEkpC93OtRu4/BLSGladQggzMBToLIQ4IIQ4KoRYKIQIphYU9mdVCNEBeBZ4PMhrKa062oBgDfACKeVxKeVZYC1aczAY3pdS/ktKWQO8D9RIKZdLKW1ob6bBzoRSymlSymmNTyCE6AWMBP4Y6VpbQLi1xqPVCtw5DyRcQlrDrbMr2qzo44ARjusNRivIgTDiWf0fYLGU8kiQ11JadbQBwRrgk26fqwh+QaxTbp+rvWwHc57JaE2HYJdSNVJrcwm31gqgQ6N9HYBgJj5tK1rDrdM5afQrUsoTUspStJrS7UFcM6xahRCDgP8A5gd5HXeUVh1sQGsG4SqhYVEBIUS31svxymTghVaeI1xaQ4GeWgsBixCiv5Ryv2PflUBLB7XailbddEopzwkhjkLI1hrVM09HAelAiRACNINiFkJcJqUc0oLzKa2tpDVuaLuBy4UQg4QQMWid5CFFCDEcSKP13g+6ahVCWBznNaM9JDFCiJa+3HTTKqWsBN4DnhVCtBdCXAv8GG1wqyW0Fa16P6tLgV8IIboIIToB/w181MJz6ak1D+iL1iQfBLwOfIw2ct8SlFYHLbUBLTbAjsGRZ4H/A/YDm1t6LgAhxOtCiNcb7f4Z8J6UslVrg4RB61NoTZRfAZMcn4PpAzRC6zQ0N5nTwCogV7bQrautaA2Dzv9Bc5MqBAqAXcCclpxbT61Syiop5UnnH1o3T42UskWriiqtrbcBajpKhUKhMIiIjYRTKBSKSx1lgBUKhcIglAFWKBQKg1AGWKFQKAxCGWCFQqEwiKB8VUebxkeUq8Rn9neFt/1tRScora3hUtDaVnSC0toa/GkFVQNWKBQKw1AGWKFQKAwiYidkVzTFNkoLgT82Ksbr8bT8Gsz5O8MpqUWUT87muwzPlllMKXRZ+IVBijwxxcVR8ugg7NHadlvIV2eedv6qHoDYD7YbrKjtEs77r2rACoVCYRCqBuyGuXNnzvyon2u7y8YT1BcVGyfIDXntII7karWbfSO8zwNtldPolR9GUS2g6j+v4bpHtzGvu2eN4rnSAWxa6L1mH07MnTtzYkJ/1uW+SA+LNvtgW8jX2Ekn2DbwA/p8OhWAzA8MFtRGCff9VwbYjbrLevDPOa+5tq1508hYXE/9kaMGqgLTwAEcf6KWfcNW+U1X070ec/8MAGz7i8IhLWiExYL9moH8dM5acjoe9zhWWFfJX0t+QBKFPr4dHsydO3Py7n7smrUIfaZ//v5h7tyZust6eOyL+vYotjOec+qYEhKoH6JVfiw7D2C/0Kr5t5qFpaem72JGZ85YY8J6/1ttgIXFgik5KRRaPLCXn0fW1ob8vP4Q9ZLCukr6WmIxCxMFOYu0t99vjTPA5uQk7AsusHvAxx77a2Udh+svurQCHBqbR9/4KQBkPd4F6uuxlZ0Nu2ZvmJKTmL9yEdZ2cU2O5eybSNKPjDO+pjhN04kJ/R2Fr4Gj9RWYLhqhKnjMyUnEWOqMluGV0jv6sf1/X/PYN+w3uXRa1mCARXQ0VaOsbHwjD4DR903BtHFX2DQWPdALgIKHmrYs9b7/rTbA9msGMn9lKJZH82TKUzPouOLLkJ/XH6Zte/jv26bw+0+XkhnVPqzX9kXaJ7XMT/sEbV3BBhacG8CGCVc10bpj5KsAnNwGL58aTck14VTbNil5VFuxZl3uizSu+Yz53ZP0WvoVdgN0BYuvZ6StUD5+MKvmzCUSWx163/9mG+DiOdn8dMwG13aXqPVeazWtxW7x67+sC7K+HsrKscnwX7sx5uQk0j6pZU7qeuJNni+DScWjKH2kBxTuZfpPHmbcW+tdzfpO5jjHf5iTup5Z227m2O3RhtaExdCB3LBsKxlRUYZp8EXxnGxeuedNAFefnztRlRJ7VVW4ZXnl9PThANyd83ety8bRaugdW0a8qe0ZX+fv+fUjK+kTFXnGF/S//802wBe71fFUyl49tHgQATbQMEwDB2BfcIH5aZ94GN/RBWMAqH+xK+127ABAbPmKt2eN4c2OWjfEd7dWArBvxHK6mNszP+1zxr17N7X16Yh5KbRbtyOsv+XiLUNJe+YATyQdRFu7MnIoeiGb2Xet5uY47813a940Mv5xnPow6/JFTbL231n+NkVgjbfohWzShpxwbY9I3tYkjRRwfOZwJt+/DoAJ8Y3XXo0MwnH/DR2Em1IyAoAvDvdh34jlRkrxjQEvgvrkWD4b8Gfcm5TXf3MXvNYZgNh1nj6ece9tcy121fGQ5itsLZhGTbd6Do3N42+O/uPRT46hOnZYWH1EK1KjWJGe77Eva9Nkhvc+xNJem8KmwxtpQ04wMaGsyf7TtkquWzmT/otLDB+AdXL2gWx+eNs3ftNc/81d9FxrnGdp8RztheYtT530+TCHATvL2ftIe8dLOQLwUcZ7bKjW3Quq1QZ4e20d9/w9t0XfTdmi1Yg6SLQFvRVeGV0wBl7rHJThdDqM98oHc/8M+sZPYcfIV+lkjuMz61quz72LaoYBxjjrX7n9PnovMJH/QBYYaIBPTx/OT1LXNdm/vbaOez+ZQdacPdQHMRJ/evpwalL0D9YovbbO9cJaeSGZt9feQEbcbkoeHcSwOK0L5czW7vT6IPzBLM7AhVfuedNna+KcrYqhGx/GOq+UsuyujLp8j880WcfPY9NbdBAcuC+KAeVW7LsLdLuGCsRQKBQKg2h2DTi+oB2jet7p2j5ckkLm1Jb1K5qutAKwL6dDi74fFiJgbqXyFT1I+mBrs79n219E1vSOjFr2IO8OfovMqPb844r3Gf0LrS/5YvVQ3fqEnfe2bKg2flxhr2Fc4d2kPiWpSbPQL/2ULtcNhIiOpnz8YH79yMomfY/51SambMgl8+FtPke9LRnpnB7Z3bXtPE+WdTId+mQDEH+8Tte+9jcPjyB91lZE1y6seWhuwyC4Ac+qt8AFb5y0Qdbjx7CdOk3sq3av3U/uacKKj3w7NDaPq7flkrRbv0s32wCnzv0C5jZsZ1Lc7IuaBg6gPjmWwvu0LohDY9/wOJ53PpWY85Hs+BNeRCsKlq38PN3uPM/qr69yDd58Zl0LwKjH74SmrfBWY+6fQUFuAtBwb4/U2zHdV4ft1F7ky+kuDeHElJBA1Sgrq+bM9Rh1316rNZunbMj1WZlwOusXTU6lIKep2+W+Ectd3Wij9oQ2X00DB9AhuTJ0JwwRLQlcEUMH0iM+MvrVg6E1ZS8YDBmEq365hvyBf26y3ybtHKyvZs3Um4nb0nT0VNFyDlcnU2GvCYu7UsGvkjh0W55ru1bWsa+uC9iNbU7UD+nncPb3NBbOMQxvxtec2BGio9n/cE8ACieH3ufdH94CcWIsdZi6doHkRMxuFsIWKzF37eL1PPays5qbZQgpvaMfO59+LXBC0HQmJ3LTsq1eB98i5RkJNxEVinywvpr/vm0KpsI9kdDyv6Q4dns049692+UREU4WnBvAhhsysJWWhv3areXYslT+PGgxyWbnExneAB1vQRZrMv/CkW12zELS1xLr2r954lzK7vU+pP/oxGmILV/pLdcnfS2x/P7TpfS2tMObO2JbfkZaQ9gN8NmPMlmZtZTGD3Le+VTWTL1ZM74hflMrwFZ2ltr6dN2vU/jWUN658TXcC1mtPcoV++/r/htFnw9zsM7TCn3jkfezH2Xy7hVvGRoV6S3IIt4Ug7Vd07RdzO3pYvZ+HmkRRnhUujALk998dH9Gwk3GkhIArGJak+6lmb/5E8/HT9RtqtSwG+Af9/ra6404XdcBseUrVfP1QqiCUsS8FEY/OUbX/tfevUoZFt1gfB8/MYTtc64mDq1Lyf3+OwNLxLwUaMFYQiiIOWFpMnGRObEjx5alGm58m0vWpsl0+Jumt2yonaK73gjwjdbR+R8nuHpWLvVxgrW/9D8I54/Gz0i4cfp6Zyy3YMXTCE+IP88zyfpdO2wG2JSQwL4XLuN/EhbSuAny0tm+/CXvRroQGRNyO8naNJmeG2uMlkHipKNUn2t9AEXpFWXVyZUAACAASURBVO0YlxJe5/d/lfUi7r1tXu//geKuAGSGOTrPg0YvN3P/DPY905EdV71KJ3PzjK8zsKh6RXeiQ/RC+UvejaxMudHrMVuMZPPEuXQxt9ee1dcsmPM1b5mUnVasZ6a50mYUlYQ8oqu+qJikomJMcXHc0uFJ7NHwwLh1QQdYOPPr20UDSXyv+V4+oaa+qJgeGzpBTqMDOjYdwmKALT17sP/hnnw5VntYnPz2zGUArHl7FKkRsBqCU6ezv6/D39q7HmgjCVUARfLNx3mm87ehlOaBe3CD896WrU8llWJEXCwf3zHf5TI1pWSEKxDHSDpnn6Bk9nDXdk23eg7dmAc0b36TKSUj2JM3EICk5aF7Zvw1fc1du1B2r6CLuemzat9dQC839yk9O/XsVVX0eF7TubziFhZZm04fZoq2uQKCnOT/OwuAzBDmV2uJOn6evn+f0kSrXqhADIVCoTAI3WvAlox0iianOtx3Gmq/L53ty5q3RwEO3+II4GJGZwonv4bRA0SWsmpu3XsHazL/4hqAaWkAhTPwwBYluN7LxCih5O6cv7uan8t2aLXKzLlfaM764/qRYGrw7f7y0yvotcT4+/6PK96HK5r/PWdgSU29VouvXtE9pDXfQHjL00jAW1l2BmtUXm+nkwGamkX5BTps7eapVceBKd0MsD/H9dUVHfnT67dERLeDO+bKOl4625dHOu0lWkTp7oTtC/uevZjHJ/HYJzcxJ3W9q9vGOXg26ZlRnKoZ7PEd5yoD7rP7A1zsEOUz8OBwSUqLAmmagzdn/dUVHYnxPV+Lfloa3d9gWV8Vxfaqvh77DlcnYxpfTXTZMYCQ9fkGS91lPTzy1KhnNRCRvsqIuX8GdakdXdttbkUMb5gSEvw6rs/935/QZVlkGV8AvtrLhglX8eNPvybT4LlrbWVnKbkGZm27mflpn3u4Iq1Iz4dV+R7pB8+ZRvc1wi3f3R3ktYfpaH0FF+wm7v3kMQAyH9avRmyK1py6vK0yYdT9lzv2sOGGDPp/cYqsqODDXR9bPNPVx9lApePPeA7VVWCqj0wL7CtY47StElHjw2cujGhBQ28Zdn1dDPC+Fy7jy7HOeOW248Zjv2YgC/70qodzu9EEG0Cx9pcvcuEJk9+AgTG/e5Luaw6QVaUNxOnZeHWuzFF5vZ1IqvnYSkvJG54NpuCHtntdiOwVMe6fPoPE9bvalAvndStnkjVHmxEtkvNWb0JugAvfGso7Ny308HZwZ8izuXT7+EBETDfXGGkREef3aSs7i+mRAYxOnsIB19wZeU3S+fLBPFRXwf3TZ9Duuzq6f7s/bM7urpU5vOhI+dbA+y+lYQ7/etHuu7qwr58YCPfVLrxhrhZhXXgzUgm5AW7siN+Y5IKaNlMAImVVDvuevZiAAeXaDGNXb9PmLvDnAP/4iSH8Y+E1mOukVjuqrdXd6K2bPZJzs+KaLDkPUCNNtN9ejO3U6Yh8+SpCi3P1Dm+rXWS8/xBZH5yLiJpv/yX1ZMVPBvC6KITeWkNqgIvnZPN079VN9q+viuLRFf8F6OMQ/n3BOTG0c3o8dwf4xnQokq5R+XA1TePe28bm+GysGT9scsxUqzXlFaGhXdEZrHlaoEVElik/lZfkHSZdJzlvDmLLV/SMalhFpjFZ75/TVWtIDfBPx2zwuhzJ9qq+9JqtDWJE3IPiRtTx8/T5dKpru39htYFqAuPuAB8pJC7fSqKPY5FQ47lUqD9ylF6ztRDaSC5TbQH3VWQao/czqwIxFAqFwiB0D8TIrzaxePt1ZGJgvH+Q2PYXkflgUeCECoXCLx2KtI6vUXvubHIs/rj3deO+j4TUAP+15AdN9i3efl2LlyxSKBRtk0RnVKDXxc6Lw6gksgmpAU76USGb8Jy7tC3UfBUKhcIIhJRtyX1boVAoLh3UIJxCoVAYhDLACoVCYRDKACsUCoVBKAOsUCgUBqEMsEKhUBiEMsAKhUJhEBFrgIUQK4QQJ4QQ3wkhCoUQUwN/yxiEEElCiPeFEJVCiMNCiJ8YrckXbSVfhRDRQojFjvy8IITYJYS4zWhdjWkrOp0IISoa/dmEEK8YrcsbQoh0IcQnQohzQoiTQoiFQoiwreTeHIQQ+UKIGrd83RfM9yLWAAPPA+lSyg7AWOA5IcRVBmvyxavARaArMBF4TQhxubGSfNJW8tUCHAFGAh2B/wesFkKkG6jJG21FJwBSynjnH9rzWg28a7AsXywCTgPdgUFoedx0yrLIYbpb/mYF9Q0ppd8/tLjBmcDXwHngHSAGuB/Y3CitBPo5Pi9Dy8BPgQpgC9AN+D1wDtgLDA50fce5soATwIRI04q29MRFINNt39vAC5GmtS3lqw8dXwN3t3WdkaIV+BlQhCMgK9K0AgXA7W7bLwFvRKjWfGBqsM+I8y/YGvAE4FagD/ADx48J9ntPASlALbAV2OnYXgO87EwohFgkhPBYPMyxr8rx408An0Sg1kzAJqUsdDvXbiCYGrDK1wBa3Y51Rcvrf18iOg3XimaAl0uHBYlArX8A7hVCxAkh0oDbgL9FqFaA54UQpUKILUKIUUFdMYi3ZDEwyW37ReB1gnujvOl27BdAgdv2FUB5ENc3A9c5MiYq0rQCI4CTjfb9F5AfaVrbUr42OmcU8H8EV/uJeJ0RorUXYAP6RKpWwAr8C23KY+k4XzC1dSO0XgMkANFoL7YLQN9AeRtsDfik2+cqgl9l8ZTb52ov2wHPI6W0SSk3Az2A3CCuGW6tFUCHRvs6oN2AQKh8DXAeIYQJrUvnIjA9iOu1FZ1g4P0HJqMZpENBXjOsWh35uQ54D62bLwVticHfRZpWACnlNinlBSllrZTyj2hdGLcHumBrBuEqgTjnhhCiWyvOFQwWoG8Lv6un1kLAIoTo77bvSoJrgnpD5WvD+QSwGG2w6G4pZUsnkm0rOiF8938y8MdWnkNPrUlAT2Chw6iVAUsJwqj5INzlSuJ3YSaN1hjg3cDlQohBQogYYHYrzuWBEKKLEOJeIUS8EMIshLgFuA/4e6RplVJWor2lnxVCtBdCXAv8GK021BJUvjbwGlozdIyUsjXrQ7UVnaC/VoQQw4E0Wu/9oGe5KgUOAblCCIsQIhGtab+7hafUs1wlCiFuEULEOLROBK5Hq8H7pcUGWGqDTs+i9XntBza39FwAQojXhRCvO0+P1iw+ijYCORf4bynlXyNQK2iuMbFoLjOrgFwpZYtqwCpfNa1CiN7AQ2juRyfd/CsnXqo69dbqxs+A96SUrVoXPgxa/xNtMO0McACtL/ixCNQaBTzn0FmK1n98p5QyoC+wmg9YoVAoDCKSAzEUCoXikkYZYIVCoTAIZYAVCoXCIJQBVigUCoMIamah0abxETVS95n9Xa/+dW1FJyitreFS0NpWdILS2hr8aQVVA1YoFArDUAZYoVAoDCIiJzdWXPqcfSCb0mv9R+z2XGsi9oPtumupvnMYAEfG2H2mSdkSBUDSkq2661F8f/jeG2DbqCEAHBsV4/V4Wn4N5vyd4ZQUENOVVorv6hQwXUwpdFn4RRgUNY/yydkMzNnD0l6b/KazHplGrw/013NmkFYMDt3ma9ZGmHL5CAC+rc8mcbkyworQoLogFAqFwiB0qQHLawdxLjM2YLpOhdq8JWLLV3rI8MvFW4ZSkRrFd7dWArBvhPfaT5Z1Mh36ZAMQf7yOdut2hE2jO+55WjbUTtFdvmtrTlZXdOTl8vtIfHcXsrZWb4l+EdHRlI8fjC1KcP30bczrHkSrIkzj2R2KtAs9fmKIT13O2vrjj1byD3O2a3/njw5gO3NGf5FtHPf7HywpO8ux7y7QUZXx6GKA9z9g4dBtrwVMd+X2+wBIPT8A+569ekjxiXy8lH8ODNy+3TdiuTblOjCpeBSnagYj6iWmbXuQ9fW66RMWC/ZrBiIt2gN7+rEadg9b1qxzTIg/zxXPvczjn92J7dRpHVQGxty5M3WX9eBihyhWzZlLn6hgp2YNH84uhe0V15A35yQ5HY/7TDuv+06Y02CkRxdNwbRRGWB/mBISqBplbfb9t74xjV4tnfusjWBoH/DuYasAuHXBHZjHJ2ErO2uknICsSM+HVfkUXKzi8Wv0NWqm5CTmr1yEtV2c1+O1so6iuqaDWMlmrTbXxdweALOQkJyIKDur6wujMaY4TfeJCf3ZNctZW/de+CrsNRypt2MWkr6WWMzCmJ6xuPe2sebUzYxYqenNiIoiWkT5/c7FDlHERkfr3sIwJSQg4vy3KuWFCuxVVYBW4zQldnQdsxt0/0VCPJXD0tn4Rh7Bz4seGZiTk8ASvImUVVqL3n4h+EnmImIQbk3mX3jsk5soucZoJW2HBecGsOGmpvOoH3hM21c4WWuB9LXE8vtPlzL9Jw+Htaun5NFBAKzLfZFABW9c4d2Y7quD5ER+/+lSMqPah0Ghd0zb9vD4NXcCcMPnB3ki6aDf9MsWvsx9s2bSccWXuura98JlfHzHfL9pxr0xkx7Pa4Ou5eMHs/Q51xJmPDpxmiH3f81Dc4kRdtqa8QVI+6SWGV0/Cjr9HR9rM2X2f3hb0N/RxQBbXzjL6OVTfB7v+vwhrTbpIN4UQ+/YMkrw7okQTg7VVXD/9BlMfPEjn03RjKgobvj8IJ/fn43csSds2pxdNl3mx2CurEOeanptc3U/z21hIjOqPdIiAk/PH0Ls7bT/PSzeC16fD3Pot0qrwVvKqrH3tHDTsq30trQLl0SvyPp6V8vm8/uzWd/++iZpyqwx7Hxae8H1iYpnxtOreD5xom4eJ4VvDeWdmxb6bA05mf/gm2y/T3sBZ8as8kgfzvtfPCebV+55EyCgZnecZa/dd9pzkVFUQvjq7Bpi6ECuW6KN8+R02uJqSQaDjLE1+3q6GGDb/iJM+70fK3xrKLO6/kOPy7aa1RUdefnZn5O4fhdvR43hzY5aU7hhoG45ANEiiieSDrJ7QU+O/XaoLgNz9vLzTHlqhsegRVfXoOWuJuNTRS9kkzbkBA+kBpyEPyykf3AOgIwuD1F01xsexzLef4gBed+5BljsgH3kYEdt03+TP5zIHXu8ugl1+7Yzg83TWPvLF+lhiWdC/HmeSdZHQ+FbQ1l6wxKGRQfOl5vj6rg5LrxjKd642K2Om+OavypTjTTRfnux6wUYLuNbPlkbVI2ddIIe8Ud5KsWZh/q3xMLWBWFKSGDfC5fxzk0Lg3qY9KR4TjZP917dZH9hTXc6rvgSidYf6Hx3V/Qern0Y4Zl+RXo+V6daSdJBo6ytDbpZWzwnm9l3rWZiQlmTY6dtlVy3cib9i46EtTbhNK7JO7LhrgYd5hpB1vvnghvdDmeVvRnYzpyh+xrBhSfczLNOWh8ctplRsb4DRC4V1ldF8djimfS6EF6PqLMPaD7pgF+/9CklWuH/8m9X+EzT86vmlzDdDHDjAAdbjOTLsXO9VulXXkjm7bU3kE54HNx/OmaDV2Pli86OjB1dMIbPrGv1ktUsLD17UPRgLwBeuedNrzWO7bV13PvJDLLm7KG+GQMDoaRTYTV9Pp2KqDGTNWcP9gsXaGxOTFdaKbzP86U8umCMK98jDUvPHux/uKdrwFMPTHFxlDw6iGFxbzY5tvJCMk9tvsu1/c6NrxleqWkNKy8k8+yaCaQ//0WTZ0NPfAUEnbNVMXTjw9hrza59zkjIXktC282kAjEUCoXCIHQLxDiSq9VePAMcGmq/L53ty8fHter8sZ3dyZhlbHhnfrWJxduvI5Om/bnO+QguVg9l0jOjPAYQw4EzaMSdit6CghzfwRj51SambMgl8+FtYa1VNEZs+YrMLdpnXzpKhyRyaKyn33j5ih4kfRCZIb8XMzo7vEy05/nxE0NcwRyhQiTEs+ahuV4Hsd48PILMBxue0/VfX8GwaO99vxX2GsYV3q0NdIZUoW/iC9rx0jXaYKA/L5KVF7SO89nvTzCk/MdOOtGk9ltYV8n4XVPJml6Erfy87hpCboBNAwdw/Ila9jl8fH2xaPNNWOeVApCx3/iCNvvgWDKn+h9Ma7duB6dqBsOqfNc+EYZorWCDRtwJ5vcYjaVnD0B7mbiTdz6VmPOhNxemhATqh/Rrsj/quFbQbPuLAp7D3LkzZ6ye3jqbXrmGTiGcH8KUkEDlsHSH+1Zg/lryAyZ0/JdX970j9XZM99VhOxW+wbnUuV+wpP0tADzxkPdKwvqqKJ5dMwHA8MqXk8K6SiZ+M4VudxbQfH+GlhFyA1z9cg27gzAWh8bm0Tdec1XLerwL1NfrH4ghBOaUFKJNgQtaWyfGUocpObzBLT4d173cW1NCAvsf7glA4WTPQvruz28hbmPwvpTBIKKjqRpldQQEeNL37/6fQ/cgiBPj+rkFluhD/ZB+XgMXztm0IItzVbF0c9uf9KNCctZPJL9RuauVdeyr6wL28M5RbkpIwBbr+5pH6yt4bPGTpD8fWRNFPXJwAp1/Vho24wsGB2LsGPkqACe3wcunRuseiGFOSSHni63cFneOSHJ30gMjglt8Oa57u7f7XriML8fOdWzp7+5TPn4wq+bMxVtAQKDn0D0IIsFkXFDB0I0PA2jN4yDSLzg3gA03ZGArLdVXWCMC3dsxv3uSXku/MrRrzBtGlBlDB+E6mePoZI7D2i6OOanr6bWtvVaL0guTICvqdMDw0sacnj6c09OHM+LrGsa/Hhl+toGIN8WEJU/F0IGM+LqGEV/XMCd1PdZ2cU3+5qSud6Vx/v359oV0Mbf38Io5VFfByIdysOw8EDJ9zns34+lVPuch8PUcnv0o06XV+VsaB5YMeTaXlI9Dp9cf9loz9lpzk77Jsx9lkpe1skn6WnuUNlGQDG8NWMbYmtxbd3796EpOPjAorJq8ETsjxhXcBOErM+5ERCgyaHMXzOj6GY9b7jRaShNqHE72DQ7aDcgQ+39evGUo8vFSzlXFknb/cWzl5xHzUrg6NTfgdxMnHfVwk9M7Ty/eMpS0Zw4EdFzvYm7vJe+avgRdjvghdJlz3rsJ8U0HVDLef4i+A443ybP5aZ8z7t27WdnXGRbdVOvR+grG/O5Juv9lf1hmQ8vaNJn+S7y75f2419eGhm+7U/RCNs9d19TH3p0J8ec5/PN1LI/R+olT5xrTFWHfs5euLw0i65HJgBZo5X7/TY/oP0lY6PuAV3THmjEtYLof3vpNwAm5Q4m5fwb7nulIN7PnfqeDdfWK7kRT7HHs+MzhVFgvMuryb7yes8+HOQzYWR6SppRzVYboX5zgM+tabcKfaM1wtlu3I6hgj+pzw7g+V/MP/ccV74dAlY/ruGkNlUeIbo74jV6QjQNCKvt0xzpIe17t1gr2jVhOvCmGvw34GF8vFE2r1oy2OSa/0RtTQTxiS/CG6qWzfflL3o10QX/j5gyykjE2nrvOMyDopbN9WbT5Jg6N9ex7fyLpIIusF3XXFgix5St6RmkxC1lM9rj/Vz53H9+VDdV1ZZaQG+DE5VtJDJDGGfrnjhY08BhZVd+GWhIAdakdOXjjUsDTrSf/31kAZDYaxT49fTiT71/n1Y3G6ahtnVca1Mh5MDhXZShoRaBH7AfbKRnkiNrzHbDTavxpHV0whgPFXV3b/dJPBRW8sr2qLz10dsR3D0xxBoTE7oa0cq0AHrEGPodRQQPe8Bes8fHxK8KyGoozKKVxkNVvz1wGwJq3R2H960n6xk9hx8hX6WRuKH+jLt8HwLeTjV1lxLniTe+6QfSpmIop2saOka+6ZmvUc2UWFYihUCgUBmFIH7A3B+j1F66gf5iDBl4625f4gobZt8ydO3PmR5qf6K8fWem13xDgpA2yHj8W2vmAG42TJJjsnBjXD0tVX7psPEF9UXHgU1w7CLu1InSafF7Ic9Pp7L+g72oO7k0lZUfDe/1gTSr4qFk6HfHfPDyCwyUpXoNgQqm18TPWdFWU5QFP9+bhEaTr7LdqKatm1B7PfntvgR7+gjXCReOgFCfLdmgtscy5X2BDc/E7uQ06uXUBOm3AqEnJEDjrW4UlI53TI7tjrpM+V4hxBg2ZEzsyatmDvDv4La1vXccxzLAbYDF0ID3ij4b7sl5ZtPkm+v3TMYnyyMGccZtm0BuFdZWsPn8Vh6uTIcSTW8c4us1WV3RkQvx5eljiXf6m1rxpZCyup/6I73wLNgBGD87Y6qmen0b+i/21mc/uCvwdd0f89FlbyWzU/64HXaK+wz6yofsr7ZkDze7D7hFfTunQgbpOQ2rfs5fomz33NR6f8MX2Wm1OEO2FFtx3Qs36qijanWwYuGxuYIkenLm+O/+c85o25eX5hikvvWEHusyH/Nf7k+lndZRQEFIDbE5Owl5R6Xd1gOuW7GgyIl5hr9GMGpWhlBOQQ2PzYGzgdE4H+PG7ptLtzgI0naHV6uyvm1vxEyb8r+dLoCBnEZkxufSb73vQwr7gArsHfBxSTcHSJyrea4CDL4xyxM/peJycVUv9pmm80khPi4l4U0Pk24r0fF5a1rfBvzbMLl5OhMUCyYnaiidu3PN3zVvGyCjIR9/+L9J/q91bzwCY4AJL9KS5z6rehNQAp31SyzevDG726gDjCu/GNL6acBvgYGmuA7webJ44l7J7ffu89bSYIAImtA+GSHXEh6YrjdhXRTk8Ihp4pNNe+n9xirzh2YYtyGm/ZiAL/vQqfS2BF781En8BMJFQrowmJAbYnJxE2ie1zEldT/7Teyh8srvPtDmd/kXj/qKa+iiiy46FQkrI6fNhjmvOinBMzpHy8QFGH5rCxQ5RLFv4sit4QHNsD/48k4pHcerXfbQFRMvCt2qHL9xXO+j+7f6wuW9lLCkBwCqmeZ28yH1lDqDJSiOmRwYwOnkKBxzTZR4am0e0iCIr6jSYjJuwWFpExPj+NuYPP32T7Xc3rMzhLQDGiHI1JCbXbxejEYTEANsrKvnmlcHkP71HG7jyMXil0fDQjC4YA4CYlwI691dFfXuUq2dpzbMnfv0nnwNsTqx504g/LBmwszxkrmbBYDtzBtPGM7Tv2oUa2TwnlaxNk+nwNy1/44/X0W6j1gwNdSPZudrF1SWBg0OcmOskieu1wY9w1nac/eYZyy1cfbip3gGNlj5vnFf2PXsxAQPKtZHEq7dp5zDXSRLLd+miuaX4C9YIJ4FW5nCuiBLuctXtL3B1bfDPrJP0ned0a62FxAA7V28ofLJ7AOOrkbVpMqaCeNeE27Hr9HFydsd25gxJS7Xm4vPtJ/JMiv/0GYtLqD9y1LBmsrxQwbg3ZmKPDv47PfNrMOfr70/pNFhJzVwy3JjeUo36omKSvHiSBHt/vf1mo36PvHYQpx+rabK/ucEaoaBd0Rkyl+eyeaL3xRactGhFlBDjbgOag542IKR9wG+vvYHF3a4LmK7/kvqwPyjuBOOgbnQ9wl5V5VrhVqFw51xmLLuHLTNaBqC1MPrOOc8PE2b4XZTS34oo32dUIIZCoVAYREhrwHo7qCsUCj8Y1Cdiv3CB/g8Hnr9Z1XybEjGzoSkUiuCIOW/nudIBTfcHv86sIkJQBlihaGPEvbeNTe819fkOx8xnitAipEGRPAqFQvF9Rw3CKRQKhUEoA6xQKBQGoQywQqFQGIQywAqFQmEQygArFAqFQUS8ARZC9BdC1AghVhitxRdCiHQhxCdCiHNCiJNCiIVCiIhz8RNCRAshFgshDgshLgghdgkhbjNalzfamNbpQogdQohaIcQyo/X4QwiR7yhPFY6/fUZrCkSk24DWlP+IN8DAq8A/jRYRgEXAaaA7MAgYCQReGjr8WIAjaPo6Av8PWC2ESDdQky/aktbjwHPAEqOFBMl0KWW84y/LaDFBEOk2oMXlP6ABFkIUCyFmCiG+FkKcF0K8I4SIEULcL4TY3CitFEL0c3xeJoRYJIT41PGm3SKE6CaE+L3jTbFXCDE4wLXvBcqBz4P5MQZq7QOsllLWSClPAn8DLo80rVLKSinlbCllsZTSLqX8CDgEXHUpaDXq/ksp35NSfgAEHYtmZLlqLm3FBrSl8u8k2BrwBOBWx4V+ANzfjO89BaQAtcBWYKdjew3wsjOhIwMWuW13AJ4FHg/yWoZpBf4A3CuEiBNCpAG3od2ESNSK27GuQCbw70tIq6E6m4lRWp8XQpQ6DM2oSNXaQhvQlsp/0AZ4gZTyuJTyLLAWrZodDO9LKf8lpawB3gdqpJTLpZQ24B3A9VaRUk6TUrpX2/8HWCylPBLktYzUuhHtjfcdcBTYAXwQoVoBEEJEASuBP0opfc+e3fa0GqazBRih9ZdABpAG5AFrhRB9CUxbsQFtqfwHbYBPun2uwtsCT9455fa52su21/MIIQYB/wHMD/I67oRbqwlYB7yHttxHCtAJ+F2kaXXi0Pw2cBGYHuQ124pWQ3S2kLBrlVJuk1JekFLWSin/CGwBbo80ra2wAW2p/LdqMp5KIM5NSCgXNh0FpAMlQgjQfrxZCHGZlHJIC86np9YkoCewUEpZC9QKIZaiDco82YLz6akVoWXoYqArcLuU0vf63IFpK1p11Rliwq1VAi1d3K6t2ICILf+t8YLYDVwuhBgkhIgBZrfiXI3JA/qiNR8GAa8DHwO3tPB8ummVUpaiDQ7lCiEsQohE4GeOa7YEPfMV4DXACoyRUla38lxtRauuOh33PQYwoxmJGNFyN0TdtAohEoUQtzj1CSEmAtej1eAiSiuhtQERW/5bbICllIVoHeT/B+wHNvv/hn+EEK8LIV53nLtKSnnS+QdUoPXJtGgNcD21OvhPtI7/M8ABtBWNHos0rUKI3sBDaA/0SdHgCzrxUtYahvv/FFoz9VfAJMfnp1pybp21RqHVzM4ApcAvgDullC3yBW4rNiCSy7+ajlKhUCgMoi0EYigUCsUliTLACoVCYRDKACsUCoVBKAOsUCgUBqEMsEKhUBhEUL6Ko03jI8pV4jP7u14dx9uKTlBaW8OloLWt6ASliGAkOwAAEHRJREFUtTX40wqqBqxQKBSGoQywQqFQGIQywAqFQmEQEbdsjtGY4uIoeXQQ9uimxzoUSRKXbw2/KC9Yevag6MFePo+HQ6vpSivFd3XCVAu9/vAV9qoqXa+nF7ZR2twux0bFeOzPWFxC/ZGjRki65GhcrtLyazDn7zRWlBcClavGtPYZUTVghUKhMIiIqgGL6GjKxw/GFqUNHKbsLMe+uyAs17ZkpHN6ZHfq4wTrcl+kh6Xp9J+PnxjC9opriHtvW1g0NcZ0pRWA0iGJVPQWFOT4XpQhHFpLhyRSkLOIo/UVjPnuSSxV2gB0OO9bKDg2Uqv5Ns7P0RumYFI14JAgEuJZ89BcrO20WSGtchq98o3V1BhLRjpFk1P9lqvGWJlGxnIL9UXFLbtmi76lE6bEjix97uWGm/TGNHq1dFLHZnLm+u78c85rji3vczjP676T52ZXsem9GK/H9cTcP4OC3AQADo19LUDq8GgVDoefHpZ4ds1qeGj7fJhDv1Xel9Cy7DyA/cIF3TR9XzAlJFA/pB/QNvO0pns95v4ZANj2FxmqxdKzB4BP47u+KortVQ2LhnSJ+o6cjscB7aU9rDiXTpeCATYCU5xm7Ovat3RO6vBQ8KskDt2W5/XYoboKaqSJbmboZHbNO020qQ5z557YSktBh1nvTPWSQ3UV9InyfGEdGpsHY71/Z+RDObTfXgyArNKm+G1rxsNoRHQ0VaOsbHxDex5GPpRD7PrdyNpajzSmxI7+T1Rbi638vJ5SfXJobB59oqYCkPmgcQbYlJDA/od7AlA42XvN99G3/4tev/3CtW0fmU3OqqUhuf733gCXPKotGbUu90X0WXVGf+6fPoP224vZNy+Ngzc2PBiPdNpL/y9OkTc8G9uZFk2l7JfEd3dx//kZLkMQDMsWvkyN1IYe7vhYmzK1/8PGdOm0VcrHD2bVnLk4n9dlC1/mvlkz6bjiS480S5972ccZNO796kG63WmMAY4U9r1wGV+OnevYah/260eMARZDB3LDsq1kREUBYM2bRsaSEup1vq69nfbfW5+vNW8a7QadY/ewVa59OZ3+xeFtN3Ps9mhsZWd1VqdR+NZQ3rnxNbT5tBs4VFfB/dNnEJdfgO3CBey1nqO30SKKrKjTYNKndi9ra4nLL2D0fVM89p9+rMYjz9xxry3/+faFANzzVi6ZU3foojEYMpaUAGAV05rV/2cUdovwyMc+UfHMeHoVhU92d+3LjFnl6srzRae41i6I0pTCt4by4LDNvL32BgDSZ3n3xOnzYQ7WeaUA2EKuInhkjI0uZu+Gd8izuSQX1JBRpJ8dCosBNg0cQPXLNa5tMS+FdusaCtzFW4aS9swBnkg6CERpxnf5cUNcgI7WVzDmd9qAUsbG43zbM8XjeBdze2Z0/YzHLXeGTVPvXqUMi/Y0vqsrOvLysz8ncf0u7G5Nz3Bjv3AB08ZdHvu61g/i6sxcn9+5fvo25nXf6fpNvXuV6qoxEM7nLP5wT4/9XZ8/xLHfDvV4Vo3i+MzhJN+s9TuOSG7aYpgQfx7ija/N9u5VylMpe+kzTmtxzTZPIONXTY1wzAmL4X2/RS9k89x1q5vsd9qA7n/Zj+3MGQ/j67RV7shW1G90N8Dy2kEcf6KW3QMbVmm+OjWXJKD6zmEARP/iBCvS8zltq+S6lTPpH07/y0aZd8FuovuaA9hOnaZk5nDuH5ofHh3NpLCmOx1XfElEBb47EFu+ImmL7+ObbdlMmdaepb02hU9UC1iRns/VqVaSjBYCVFgv8s3AoFY6B+Cls31Zssb/8mkdiiTRFLdSWQPFc7J5urdm0CYmlAHw5pAT3hNHwJBL2pATLp1O1ldF8djiJ+m19CtsDr/2sw9kU3qtth5sv/RTrEjPD5kG3Qyw07n9SG49+9yao1duv4+uhdWa8c3V3pKfWdcCUGYT9Jt/kPpTp/WSFZBks+TAY30x1/TjgXHrHLVyYylbn0qfI1M99rU7GUU6kREU0lwSl28lf+RQcBjgO1K/4U/Tb6HLwi8CfPP7SfnkbEZdvsdvmtEFYzhQ3NW1HV/Qjl5zw5ufPx2zoYlB80Xn7BOuCljsB9v1lNVs7NFQ/OQg1/YPb/3GZ2XBac9aigrEUCgUCoPQpQYsrx3EkVyt52TfiOUAVNhrGFd4N6lPSex7vuLI4qEcuuJ913cK6yq596uppNUe10OSH7Gem13M7Smc7NvP1gidqUHUZC7eMpR+6ac89hmWpwForPXq2EMsGnSRLgZqimRiJ50I2F1TvqIHmUvbTotoTNrXLO+ndZHEGqzFnZvj6po1ENtlfgxiy67ACX0QcgNsGjiA40/UenQ7nLZVMuv4zZjGV2MrO4Zp4AA6JFd6fG/1+avodmeBoSOivtheq/X/rL9wBX8t+UHE6RRDB5L2zIEmfVORmqfy8VJXtxPA7INjDfWCcCK8dKh72xdOxNCB9IgPPB5S0VuQOHIwUce1gTijB7gCseTdW8LeRdKYwyUpbO9f12SA2x+FdZWsPn+Va9tcWdeqcZjQGGAhMKekgElgX3CB3QM+9jj8x/M/4NjY9mABc9cuXtNEMvf8XRvRz3xwB0kUGqzGDUe+37Bsa5O+6gp7DYerk4FK7981CHNyEjGWOqNltBl6vnqI+WmfA/4jGgtyFkEO9P275hKYNb2jYUEW7sRY6jAlR8IwZlMyp+7gnsW5HLrtrYBpz9m0Abnxu6bS7U73MHv/ffOBCIkBNqekkPPFVrKiTtPTYqLxw/JIp738aNs3rm1vaRTNx5nvt8Wdo7GP8LjCuzGNrybSDHDaJ7XMT/sEdf+D49jt0Yx7927+FmSFZcfIVwEYtSwygizWZP6Fxz65iWNjIybkoEUM3fgwAFnTi0Laomx1rjgDKG6LO0e08O74HS2isLYLvpofLornZPPKPW/6TRMpDuON8cz3pnlbUx9FdNkxA5R5x5ycRNontcxJXU+8SXN8v3L7fQDauICR4vww8zd/4vn4iYZ5aNjKzmJ6ZACjk6f4Tdf1+UOsSM93haLrEWTREuJNMcxJXc8fP/+BK8gK4A8/fZNftNM8e3wFa+iNrwAnb6y6Vov2vGduaIOGWm2Abe2jXAEUABnvP0TyjqbOFfVxgrW/9DPL2JyriSN8IalFL2Qz+67V3Bznvzk87brPWX5AGyxInWtsv1r55GxAG5TpEX/UI9+djC4YA2jBLoTQx7M1mAYOwL7gAvPTPnEZ36xNk+m9QHtO7Hu+MlKeC28O9RPiz/NMcvi1uGPfszegu9LRivRwSPHLutkj+WvHG0icdNSjj7+LuX2TZ/XmuDqeHqf5DL959QgOl6SEdRyg8K2hLL1hidcAp5ee/wkAT/z6T1qAC+gWNNRqA9yu6AzWvGmu7az3z3lMRSivHcT+ByyYom20F9pj9P/bO/eYpu4ojn/bUko7kMdKHYyHFi3USHTDR+Yjiks0i5nTRMkcxmgy0SFx8UEW4zKNCXHxuRAfk0xH3NDNOGc0hMhm0CFGDUNxRlRG8b2Jw861Fkpf+6PctrdcXvb23qs5n//u7U366/3dnnt+v3O+52TWLoa8yW+Ih5g8iDsu7FuQKwmbi+KEFuw1dgkwIjbymBjc+nIUPFF+v5vJBe0rIj5F690LPpadiuTT4R3jQPAJcbIqwWw7ZNYuRuq+CMjqpFWQW9vwL4afLPAWEwpEAqKBlwHN8UvQAOgwT4BxbCHrM1eUB+fzt7Nkv7+aRwEAHl9IRpxAYkjmf/Xju7s5g2+3O5OQ0J1NsuW1fNxdwdYCLEuvxeaSPN689pANsPP+A6Rt8kdpg5eSZoMare8xaV0ajLm8EOmlcsjqpJl0b3bZMO7cShyZXDao6CifRKSmoHllKi7O2d6rTr03NibeAABolthxMHqW6N0qzAY1GieU+47989+78W0rmoTObgV44lVvOqMQyfruxiYYd+iREb0U9dP2sCrLSZm2okn4SApv227UJy4jLUi0pxiqQ/uHMugU3uOl96bietloAEDaQeFsgUyjRuXsXZx1Miosr+O7U7k+gZNu9wVUzslmGeD8mHa0vl+D2g38xDBIiEEQBCESYQtNMt0b2sexfeJQE5fDyW3Hcyy48jEyi0yo/i0bE1Q3RRlHlz6xWwzy4uXxihNaUFywF01dNqw9OBcQ2AMezPwz3UgY1q+qQF70M2x7moFDVmGT9V3NJmSu1eHvS0B8t7fmNlrhmeyVpsrqpLFfDfg7yDD3C/B6cQDwsCEJeonEALi4WJUtqOc7EBqs6dD93nc4+GyHHAcuT4EB/OxXh8UAs7s37AcA2D0OlJqzQk5cDidnbSOh2xUFxxg9dMpm0caheO7AtqcZWBV/kzPDAfCKW8rMOaxzM2P+EG3bJJg7c+MBAKZ5XlURM//PhkUhFuxuGaZcdQ/10VFrLA5/PQvJEqgPcWvqIYxRdWdsPMuC+3p4X8yB3S4AQHnjAauesyIxEY5RKegaosSRku2s0pSfn58HADBwVCAj+ia4iwyXCIZv0RDvBlgRF4tbG2PROsMfyLB7HKiyxaMmVw/Pk9ASl8NJQewj3irdh8TVm6jJy8EHVddgUPY0qFZ3JzY8mol7E9k5vgcGmFQuBnedXajJy8GCo9X9FjhqdVixc/MK6L4X3/gyMPWNp++cC9XM8H6X8+0R+CXgOXyrpBBJx/yRwL/mjwhoAeU3vm2u55B1KsI7uFccldwBxVBvXe1AgRMjxDDb1HiDx+/j3QA/LE9Gfc4eAP5N7lJzFmpy9d7WOES/uCeORunhPciI4F54S1Vk0RcZEWp8VfUt0iMi0V/e5ZKiNYirviLZlZLQnPpsKyzF/nBNjNwNru4tUyrWIbPE6+BINa9a6gSKxrx5y95nVbJCjGDiNR2s6PGiO9Pxz6oUSXu+fSGGEENxrQXLilajfPdO1vIyULjgGqDIQq9UIvdMC84seQeeevHmQCGTw6Dk3tNedGc6Hq8f7jvWNDSJW2S+/Sk+zS/E/G+qfc0XhSSi4U9MW17gm3+u3PlAmM4NI0334aT+eiHBJRpj2QCe5d38SJHjYvGwPBnxmg5syjgJwC8IcG4dish68QutBCPbocX4ZH/XhvZxbpjm7e9xnRiV+90WC9TVjVi4YR1cSv/Sk6k7OhjhwhOXHYf3zULS3WZBlXzDTpgBAOPvfQJruoy1xxss1ol+5EDkOf8zIrb35nE6Iau7ijbHEADCG+DA+V/zxRFfgC2Yvjo3EPyg/3k5ssr+C5sN4McDVqnww9gDrNw6pji0QQLtXLiIPF3P6nSgbTDC+KSwx3VvnuvscU4IPHY7q8niQEg9JYfxPvs3yO1gVfcXCkaMk9AI6FJTYETvYh2p8lPZDFRoZ7DO8d1FojeY+d8Sl4+NWu5rxJrbF8FjsWL+/nVwq7zHov2vgsbRH+F+VnkxwB5bB2ZXrmaptrR10ojGDxR3YxPSGsUeRWhwJcAD4nuU/Yl1pIoUOnT0N4aX5V66bTakbBH/fg52HOG+vyTEIAiCEAlePGC3xYKRK4UrpEMQBPEqQB4wQRCESJABJgiCEAmZx0Pp7gRBEGJAHjBBEIRIkAEmCIIQCTLABEEQIkEGmCAIQiTIABMEQYgEGWCCIAiR+B8ybcKkobmJjQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "count = 1\n",
    "for i in images:\n",
    "    plt.subplot(4, 8, count) \n",
    "    plt.imshow(np.squeeze(i))\n",
    "    plt.title('num:%s'%labels[count-1])\n",
    "    plt.xticks([])\n",
    "    count += 1\n",
    "    plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过上述查询操作，看到经过变换后的图片，数据集内分成了1875组数据，每组数据中含有32张图片，每张图片像数值为32×32，数据全部准备好后，就可以进行下一步的数据训练了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 构造神经网络"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在对手写字体识别上，通常采用卷积神经网络架构（CNN）进行学习预测，最经典的属1998年由Yann LeCun创建的LeNet5架构，<br/>其中分为：<br/>1、输入层；<br/>2、卷积层C1；<br/>3、池化层S2；<br/>4、卷积层C3；<br/>5、池化层S4；<br/>6、全连接F6；<br/>7、全连接；<br/>8、全连接OUTPUT。<br/>结构示意如下图："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LeNet5结构图"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"https://gitee.com/mindspore/docs/raw/master/tutorials/training/source_zh_cn/quick_start/images/LeNet_5.jpg\" alt=\"LeNet5\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "更多的LeNet网络的介绍不在此赘述，希望详细了解LeNet网络，可以查询<http://yann.lecun.com/exdb/lenet/>。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在构建LeNet5前，我们需要对全连接层以及卷积层进行初始化。\n",
    "\n",
    "`Normal`：参数初始化方法，MindSpore支持`TruncatedNormal`、`Normal`、`Uniform`等多种参数初始化方法，具体可以参考MindSpore API的`mindspore.common.initializer`模块说明。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "使用MindSpore定义神经网络需要继承`mindspore.nn.Cell`，`Cell`是所有神经网络（`Conv2d`等）的基类。\n",
    "\n",
    "神经网络的各层需要预先在`__init__`方法中定义，然后通过定义`construct`方法来完成神经网络的前向构造，按照LeNet5的网络结构，定义网络各层如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:47.178650Z",
     "start_time": "2020-10-20T07:21:47.143316Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "layer conv1: Conv2d<input_channels=1, output_channels=6, kernel_size=(5, 5),stride=(1, 1),  pad_mode=valid, padding=0, dilation=(1, 1), group=1, has_bias=False,weight_init=normal, bias_init=zeros>\n",
      "****************************************\n",
      "layer fc1: Dense<in_channels=400, out_channels=120, weight=Parameter (name=fc1.weight, value=[[-0.00758117 -0.01498233  0.01308791 ...  0.03045311 -0.00079244\n",
      "  -0.01519072]\n",
      " [-0.00077699 -0.01607893 -0.00215094 ... -0.00235667 -0.01918699\n",
      "  -0.00828544]\n",
      " [-0.00105981 -0.01547002 -0.01332507 ...  0.01294748  0.00878882\n",
      "   0.01031067]\n",
      " ...\n",
      " [ 0.01414873 -0.02673322  0.01534838 ...  0.00437457 -0.01688845\n",
      "  -0.00188475]\n",
      " [ 0.01756713 -0.0201801  -0.0223504  ...  0.00682346 -0.00856738\n",
      "   0.00753205]\n",
      " [-0.01119993  0.01894077 -0.02048291 ...  0.03681218 -0.01461048\n",
      "   0.0045935 ]]), has_bias=True, bias=Parameter (name=fc1.bias, value=[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.\n",
      " 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.])>\n"
     ]
    }
   ],
   "source": [
    "import mindspore.nn as nn\n",
    "from mindspore.common.initializer import Normal\n",
    "\n",
    "class LeNet5(nn.Cell):\n",
    "    \"\"\"Lenet network structure.\"\"\"\n",
    "    # define the operator required\n",
    "    def __init__(self, num_class=10, num_channel=1):\n",
    "        super(LeNet5, self).__init__()\n",
    "        self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid')\n",
    "        self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid')\n",
    "        self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02))\n",
    "        self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02))\n",
    "        self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02))\n",
    "        self.relu = nn.ReLU()\n",
    "        self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2)\n",
    "        self.flatten = nn.Flatten()\n",
    "\n",
    "    # use the preceding operators to construct networks\n",
    "    def construct(self, x):\n",
    "        x = self.max_pool2d(self.relu(self.conv1(x)))\n",
    "        x = self.max_pool2d(self.relu(self.conv2(x)))\n",
    "        x = self.flatten(x)\n",
    "        x = self.relu(self.fc1(x))\n",
    "        x = self.relu(self.fc2(x))\n",
    "        x = self.fc3(x) \n",
    "        return x\n",
    "    \n",
    "network = LeNet5()\n",
    "print(\"layer conv1:\", network.conv1)\n",
    "print(\"*\"*40)\n",
    "print(\"layer fc1:\", network.fc1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建完成后，可以使用`print(LeNet5())`将神经网络中的各层参数全部打印出来，也可以使用`LeNet().{layer名称}`打印相应的参数信息。本例选择打印第一个卷积层和第一个全连接层的相应参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 自定义回调函数收集模型的损失值和精度值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "自定义一个收集每一步训练的`step`，每训练一个`step`模型对应的`loss`值，每训练25个`step`模型对应的验证精度值`acc`的类`StepLossAccInfo`，该类继承了`Callback`类，可以自定义训练过程中的处理措施，非常方便，等训练完成后，可将数据绘图查看`step`与`loss`的变化情况，step与acc的变化情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参数解释如下：\n",
    "\n",
    "- `model`：函数的模型Model。\n",
    "- `eval_dataset`：验证数据集。\n",
    "- `step_loss`：收集step和loss值的字典，数据格式`{\"step\": [], \"loss_value\": []}`。\n",
    "- `steps_eval`：收集step和模型精度值的字典，数据格式为`{\"step\": [], \"acc\": []}`。\n",
    "\n",
    "以下代码在会作为回调函数，在模型训练函数`model.train`中调用，用于收集训练过程中step数和相对应的loss值、精度值等信息，最终使用收集到的信息，进行可视化展示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:21:47.246860Z",
     "start_time": "2020-10-20T07:21:47.180702Z"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore.train.callback import Callback\n",
    "\n",
    "# custom callback function\n",
    "class StepLossAccInfo(Callback):\n",
    "    def __init__(self, model, eval_dataset, step_loss, steps_eval):\n",
    "        self.model = model\n",
    "        self.eval_dataset = eval_dataset\n",
    "        self.step_loss = step_loss\n",
    "        self.steps_eval = steps_eval\n",
    "        \n",
    "    def step_end(self, run_context):\n",
    "        cb_params = run_context.original_args()\n",
    "        cur_epoch = cb_params.cur_epoch_num\n",
    "        cur_step = (cur_epoch-1)*1875 + cb_params.cur_step_num\n",
    "        self.step_loss[\"loss_value\"].append(str(cb_params.net_outputs))\n",
    "        self.step_loss[\"step\"].append(str(cur_step))\n",
    "        if cur_step % 125 == 0:\n",
    "            acc = self.model.eval(self.eval_dataset, dataset_sink_mode=False)\n",
    "            self.steps_eval[\"step\"].append(cur_step)\n",
    "            self.steps_eval[\"acc\"].append(acc[\"Accuracy\"])\n",
    "            "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 搭建训练网络并进行训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建完成神经网络后，就可以着手进行训练网络的构建，模型训练函数为`Model.train`，参数主要包含：\n",
    "1. 每个`epoch`需要遍历完成图片的`batch`数：`epoch_size`；\n",
    "2. 数据集`ds_train`；\n",
    "3. 回调函数`callbacks`包含`ModelCheckpoint`、`LossMonitor`和`Callback`模型检测参数；\n",
    "4. 数据下沉模式`dataset_sink_mode`，此参数默认`True`需设置成`False`，因为此功能不支持CPU模式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 定义损失函数及优化器\n",
    "\n",
    "在进行定义之前，先简单介绍损失函数及优化器的概念。\n",
    "\n",
    "损失函数：又叫目标函数，用于衡量预测值与实际值差异的程度。深度学习通过不停地迭代来缩小损失函数的值。定义一个好的损失函数，可以有效提高模型的性能。\n",
    "\n",
    "优化器：用于最小化损失函数，从而在训练过程中改进模型。\n",
    "\n",
    "定义了损失函数后，可以得到损失函数关于权重的梯度。梯度用于指示优化器优化权重的方向，以提高模型性能。\n",
    "\n",
    "MindSpore支持的损失函数有`SoftmaxCrossEntropyWithLogits`、`L1Loss`、`MSELoss`等。这里使用`SoftmaxCrossEntropyWithLogits`损失函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:17.681717Z",
     "start_time": "2020-10-20T07:21:47.248874Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 1 step: 125, loss is 2.3026962\n",
      "epoch: 1 step: 250, loss is 2.315806\n",
      "epoch: 1 step: 375, loss is 2.306828\n",
      "epoch: 1 step: 500, loss is 2.306708\n",
      "epoch: 1 step: 625, loss is 2.3070762\n",
      "epoch: 1 step: 750, loss is 2.2980754\n",
      "epoch: 1 step: 875, loss is 2.255021\n",
      "epoch: 1 step: 1000, loss is 0.76986873\n",
      "epoch: 1 step: 1125, loss is 0.24157292\n",
      "epoch: 1 step: 1250, loss is 0.19213136\n",
      "epoch: 1 step: 1375, loss is 0.21492077\n",
      "epoch: 1 step: 1500, loss is 0.29394197\n",
      "epoch: 1 step: 1625, loss is 0.099394925\n",
      "epoch: 1 step: 1750, loss is 0.2307421\n",
      "epoch: 1 step: 1875, loss is 0.16296464\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from mindspore import Tensor\n",
    "from mindspore.train.serialization import load_checkpoint, load_param_into_net\n",
    "from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor\n",
    "from mindspore.train import Model\n",
    "from mindspore.nn.metrics import Accuracy\n",
    "from mindspore.nn.loss import SoftmaxCrossEntropyWithLogits\n",
    "\n",
    "if os.name == \"nt\":\n",
    "    # clean up old run files before in Windows\n",
    "    os.system('del/f/s/q *.ckpt *.meta')\n",
    "else:\n",
    "    # clean up old run files before in Linux\n",
    "    os.system('rm -f *.ckpt *.meta *.pb')\n",
    "\n",
    "lr = 0.01\n",
    "momentum = 0.9 \n",
    "\n",
    "# create the network\n",
    "network = LeNet5()\n",
    "\n",
    "# define the optimizer\n",
    "net_opt = nn.Momentum(network.trainable_params(), lr, momentum)\n",
    "\n",
    "# define the loss function\n",
    "net_loss = SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')\n",
    "\n",
    "# define the model\n",
    "model = Model(network, net_loss, net_opt, metrics={\"Accuracy\": Accuracy()} )\n",
    "\n",
    "epoch_size = 1\n",
    "mnist_path = \"./datasets/MNIST_Data\"\n",
    "# save the network model and parameters for subsequence fine-tuning\n",
    "config_ck = CheckpointConfig(save_checkpoint_steps=375, keep_checkpoint_max=16)\n",
    "# group layers into an object with training and evaluation features\n",
    "ckpoint_cb = ModelCheckpoint(prefix=\"checkpoint_lenet\", directory=\"./models/ckpt/quick_start/\", config=config_ck)\n",
    "\n",
    "ds_dataset = create_dataset(\"./datasets/MNIST_Data/train\")\n",
    "eval_dataset = create_dataset(\"./datasets/MNIST_Data/test\")\n",
    "\n",
    "step_loss = {\"step\": [], \"loss_value\": []}\n",
    "steps_eval = {\"step\": [], \"acc\": []}\n",
    "# collect the steps,loss and accuracy informations\n",
    "step_loss_acc_info = StepLossAccInfo(model , eval_dataset, step_loss, steps_eval)\n",
    "\n",
    "repeat_size = 1\n",
    "ds_dataset = create_dataset(\"./datasets/MNIST_Data/train\")\n",
    "eval_dataset = create_dataset(\"./datasets/MNIST_Data/test\")\n",
    "ds_train = create_dataset(os.path.join(mnist_path, \"train\"), 32, repeat_size)\n",
    "\n",
    "model.train(epoch_size, ds_train, callbacks=[ckpoint_cb, LossMonitor(125), step_loss_acc_info], dataset_sink_mode=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "参数解释：\n",
    "\n",
    "`ModelCheckpoint`：模型保存函数，作为回调函数调用，用于设置保存模型的名称、路径、保存频次和保存模型的数量等信息。\n",
    "\n",
    "训练完成后，能在Jupyter的工作路径上生成多个模型文件，名称具体含义`checkpoint_{网络名称}-{第几个epoch}_{第几个step}.ckpt`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看损失函数随着训练步数的变化情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:17.822724Z",
     "start_time": "2020-10-20T07:22:17.682710Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd7wU9b3/8ddHQNSIlIBGFMSCxhIbqBh/N/FejSJJLNFYoqImijVqNDca9RpvlFhjjEbFgvUqImIhsWAXu4KCigVRkV4FAQsK5/P74zubnd2z/eyeLef9fDz2sbMz35n57Ow585nvd2a+Y+6OiIi0batVOwAREak+JQMREVEyEBERJQMREUHJQEREUDIQERGUDKQMzOwiM1toZnNbeb3DzOx/WnOd+ZjZNDPbsxXX18fM3Mzat9Y6yy2Kf7Nqx9HWKRk0iNbeCcXW2ws4E9jK3b9XwfUcbWYvxMe5+wnufmGl1in1qVr/C/VOyUBaaiNgkbvPr3Yg0rbVc+2oFigZtAFmdpyZTTWzz8xsjJn1jMabmf3NzOab2edm9paZbRNNG2Rm75rZMjObZWa/z7DcPYEngJ5mttzMbjOz3c1sZlq5fx+pmdkFZnavmd0RLXuymfWPle1lZveb2QIzW2Rm/zCzLYFhwK7RepZEZW8zs4vyfc9ompvZCWb2oZktNrNrzcwyfKeeZvaVmXWLjdshagbrYGabmtnTUWwLzewuM+uSZbunx5eybaJ1jY6+6ydmdmqO33BNM/urmX0a/VYvmNmasSKHm9n0KKZzY/PtbGYvm9kSM5sTbc/VC9kuZtYuWufCKL5T4k1SZtbZzIZHy51lobmwXZb425nZOWb2UfS7T4hqlQl7Zokh5/aO/rbOMrO3gC/MbATQG/hn9Lfyh2zbVNK4u14N8AKmAXtmGP9fwEJgR6AjcA0wLpq2NzAB6AIYsCWwfjRtDvAf0XBXYMcs690dmJntc3pswAXA18AgoB1wMfBKNK0dMAn4G/AdYA3g/0XTjgZeSFvubcBF+b5nNN2Bf0XftTewABiY5Ts9DRwX+3w5MCwa3gz4SbSOHsA44Kos3/Xf8aVvG8KB2ATgfGB1YBPgY2DvLDFdCzwLbBBtpx9GMfSJvttNwJrAdsAKYMtovn7AAKB9VPY94PRCtgtwAvAusGH0N/BkVL59NP1B4Ibot1oXeA04Pkv8/w28DWxB+FvbDvhuATEUsr0nAr2ANXP9L+iVZx9S7QD0KtMPmT0ZDAcui31eG/g22jH8FzAl2lmsljbfdOB4YJ086/33Di7T5/TYCMngydi0rYCvouFdox1B+wzrOZrcySDr94w+O1FiiT7fC5yd5TsdCzwdDRswA/hRlrL7A29m+a7/ji992wC7ANPTlvVH4NYM61gN+ArYLsO0PtF32zA27jXg0Czxng48EPucdbsQkuLxsWl7RuXbA+sRks6asemHAc9kWe8HwH5ZphXz22Ta3r8u5H9Br9wvNRM1vp7Ap4kP7r4cWARs4O5PA/8gHHXOM7MbzWydqOiBhKP3T83sOTPbtYwxxa86+hJYI2p66AV86u4rS1hm1u+ZY71rZ1nWfYQmqZ7Ajwg7q+cBzGxdM7snahZZCvwf0L2EeDciNK8tSbyAcwg72XTdCbWkj3IsL+N3M7PNzexfZjY3ivcvGeLNtl16EhJhQnx4I6ADMCcW/w2EGkImvUqMv5DtPQNpMSWDxjeb8I8LgJl9B/guMAvA3a92937A1sDmhOo87v66u+9H+Od+kHC0VogvgLVi62tHqN4XYgbQ2zKfCMzXvW7O71kMd18CPA4cDPwKGOHRISehWcuBbd19HeAIQu0hk5RtAcSvtpoBfOLuXWKvTu4+KMNyFhKa1jYt9rsA1wPvA32jeM/JEW+6OYQmooR4G/8MQs2geyz+ddx96yzLmkFp8ReyvdP/NtQVcwmUDBpLBzNbI/ZqD9wNHGNm25tZR8KR4avuPs3MdjKzXcysA2HH9TWwysxWN7PDzayzu38LLAVWFRjDFMKR/k+j5Z5HaO8txGuEHdAlZvad6DvsFk2bB2wYP/mZJuv3LHDdmZY3mFBDujs2vhOwHFhiZhsQJc8sJgKDzKybmX2P0EST8BqwNDr5uWZ0gnUbM9spfSHu3gTcAlwZnXRuZ2a7Rt8zn06E32+5mX0fOLGAeRLuBU4zsw2ik7ZnxWKaQ0iYfzWzdcxstehk74+zLOtm4EIz62vBtmb23QLjL3R7J8wjnIORIigZNJZHCG3LidcF7v4U8D/AaMKOdlPg0Kj8OoQTj4sJTSyLgCuiaUcC06Kq+QmEI7K83P1z4CTCP/8sQpKZmXOm5LyrgJ8TThpOj+Y7JJr8NDAZmGtmCzPMm+t7lmIM0BeY5+6TYuP/l3CS+nPgYeD+HMu4k3BCfBphxzkyFm/iu24PfEI4+r8Z6JxlWb8nnIB9HfgMuJTC/n9/T6jdLCP81iNzF09xUxT3W8CbhL+vlSQPDAYTTn6/S/gbug9YP8uyriQkl8cJyWk44YR3PsVs74SLgfOi5qtmV8FJZpas/YqIZGdm+xCuqtoob2GpO6oZiEhGUfPVIDNrHzXR/Al4oNpxSWWoZiAiGZnZWsBzwPcJzY4PA6e5+9KqBiYVoWQgIiJqJhIRkXAnYd3p3r279+nTp9phiIjUlQkTJix094z3/dRlMujTpw/jx4+vdhgiInXFzD7NNk3NRCIiomQgIiJKBiIigpKBiIigZCAiIigZiIgISgYiIoKSQeuYOBFeeqn11tfU1PJlfPMNLFmS/Lx8OSxs1nN0qs8+g48+gqefbvn6m5rg449zl3GHu++GL75o+fpE2jglg3zeeiu8f/lleBVq/vywc5w/H3bYAXbbDaZNC9OWLoWHHoLDDoPnn0+db8mSsJMDWLwYVqwIw598AjNmwL33huXGrVgBZ5wBL74I++8P7drBqlWwaBHMnh3KvP8+HH88rIyeKLloUXL+TP1THXggdO0ahu+6Czp1gh49YNQo+PpruO66sCMGmDcP9twTvvtd2Gwz2GOPsK1++cuQHFatgnvugYsugq++Sl1vPHENGgSrrw7vvRe+w6abwhNPpMY1dSosWBCGX34ZDj8cTj89tczdd8Pvo27sH34YzOD118M2eOwxmD49+Z0feihMN4Nnnmm+HUTaimo/hLmUV79+/bwsPvnEffBg98MOcx81yn3LLd0328x9nXXchwxxD7uM3K/rrnN/8skwvNlm7iNHul96afbyhxxS2HITr623dr/qqubjL788OTx4cHHLTLxOOsl99OgwvOmm4f2ee9wvuihZZqedSlt2//6Zx6+1VngfOjQ57ve/z72skSObjzvyyNRt+dvfunfqlFqme/fiYt5kE/fjjnM/44zk38iECe7z55fn702kyoDxnmW/Wpe9lvbv399L6o7iuuvgzDPDka1ILmuskfw76dEj1Cq6datuTCItZGYT3L1/pmltq5lou+3gt7+Fo4+udiRS6+IHDAsWhCawsWOrF49IhbWtZLDbbnDZZXDrrcn26oUL4e9/h/XXD0d/110HM2fCb34DHTqEd4Af/QimTIE334RbboFvvw0nhS+6KLRZX3hhcj3HHNN83WedBZtsEtrUH3ooOX7LLZPDxx+fPfY99oBJk2CttWDMmMxlbr4ZZs2Cfv3C5759ww7s9dfh0UfhBz8IJ3dvvx1OOSV13uuvh5tuyr7+rbZqHk+6xDadNw9Gjw470auvTi2z227w6183n/e446BjR3j22fB54MCw7TfeOLXcL38Jf/oTHFrA44233jp/mYSePcNvf+SR2csMHFj48kTqTbb2o1p+le2cQSG+/db9hBPcp07NX/bOO91feil1XKI9OtNyx40Lw01N7l98EYYPPND9D38I08D9wQebz7t4cZh2wgnuJ5/s/tln7nPmpJZZtSp/vLNmuX/zTVh/uttuc3/00XD+Y+bMMG7+fPfNN3d/7bXkPInzDXPn5l7Xxx+HuBPf94Yb3Jcvd583z338+NzznnZaiGXFitTxS5a4f/VVWN5dd7lff31ye194YSjz/PPuH3yQjBXczzzT/dBD3SdPdt955zDu+eeTy7399nA+4ogjmp9XmDIlLDvTNhOpceicQRW9+mq4KmfzzYufd9Gi0DxRy955B3r1gs6dqx1JMHs2PPAAnHRSuEIobvjwcLVS/Oh/5sxQUzzvvOblIdRu1l23+fg334Ttty9v7CIVluucgZKBSD7Tp8NGG6WOe/tt2Gab6sQjUiKdQBZpid69wytuNf3rSGPRX7RIIT79NFwEkPDNN9WLRaQClAxECnXJJcnht9+uXhwiFaBkIFKKwYOrHYFIWSkZiBTjjDOqHYFIRSgZiBTjvPPCezE3tInUASUDkWJ07RruqejSpdqRiJSVkoFIsX74Q11NJA1HyUCkWKuvrmQgDUfJQKRYq68eOg1MPHhIpAEoGYgUa+LE8H7ppdWNQ6SMlAxEivXhh+F9+fLqxiFSRkoGIqX63veqHYFI2SgZiBRr+PDwfsUV1Y1DpIyUDESK9atfhfc5c8LzE0QagJKBSLHat692BCJlp2QgUqx27ZLDeq6BNIiK/iWbWS8ze8bM3jOzyWZ2WoYyZmZXm9lUM3vLzHasZEwiLRZ/PKaSgTSIStd3VwJnuvsbZtYJmGBmT7j7u7Ey+wB9o9cuwPXRu0jty/TcZJE6VNHDGnef4+5vRMPLgPeADdKK7Qfc4cErQBczW7+ScYmUTR0+Q1wkk1ar45pZH2AH4NW0SRsAM2KfZ9I8YWBmQ8xsvJmNX7BgQaXCFClOUxP84Q9w443VjkSkRVolGZjZ2sBo4HR3X5o+OcMszQ633P1Gd+/v7v179OhRiTBFiucOl18Oxx9f7UhEWqTiycDMOhASwV3ufn+GIjOBXrHPGwK6eFvqg5qJpEFU+moiA4YD77n7lVmKjQEGR1cVDQA+d/c5lYxLpGyampLDU6dWLw6RFqr01US7AUcCb5tZ1NUj5wC9Adx9GPAIMAiYCnwJHFPhmETKJ14z6NtXNQWpWxVNBu7+ApnPCcTLOHByJeMQqRjt/KVB6I4ZkVIcdVR4jzcTidQxJQORUvznf4Z31QykQSgZiJQicedxojtrkTqnZCBSikQyGDq0unGIlImSgUgp1EGdNBj9RYuUQh3USYNRMhAphWoG0mD0Fy0iIkoGIiV5+eVqRyBSVkoGIqVYtaraEYiUlZKBSCl0s5k0GCUDkVKoGwppMEoGIiKiZCBSEjUTSYNRMhAphZKBNBglAxERUTIQKYlqBtJglAxESqFkIA1GyUCkFN//fubx8+frhjSpS0oGIqUYMiTz+PXWg/POa91YRMpAyUCkFO3aZZ/20EOtF4dImSgZiJRCzzOQBqNkIFIKPc9AGoz+okVKoWQgDUZ/0SKlyNVMpMtOpQ4pGYiUQjUDaTD6ixYphU4gS4NRMhAphZKBNBglA5Fy0zkDqUNKBiIiomQgIiJKBiLlN3t2tSMQKZqSgUi5LVsGTz5Z7ShEiqJkIFIJb75Z7QhEilLRZGBmt5jZfDN7J8v03c3sczObGL3Or2Q8ImX1ve9ln6ZLT6XOVLpmcBswME+Z5919++j15wrHI1I+kyfD9ddnnqZkIHWmosnA3ccBn1VyHSJV060bDMxyrKNkIHWmFs4Z7Gpmk8zsUTPbOlshMxtiZuPNbPyCBQtaMz6R7LLt9JUMpM5UOxm8AWzk7tsB1wAPZivo7je6e39379+jR49WC1AkJ+30pUFUNRm4+1J3Xx4NPwJ0MLPu1YxJpCjZei9VkpA6U9VkYGbfMwv/NWa2cxTPomrGJFIU7fSlQbQvtKCZrQf8Bejp7vuY2VbAru4+PMc8I4Ddge5mNhP4E9ABwN2HAQcBJ5rZSuAr4FB39fIldUTnDKRBFJwMCJeJ3gqcG32eAowEsiYDdz8s1wLd/R/AP4qIQaS2qJlIGkQxzUTd3f1eoAnA3VcCqyoSlUi9UM1AGkQxyeALM/su4ABmNgD4vCJRidQLJQNpEMU0E50BjAE2NbMXgR6ENn+RtkvPQpYGUXAycPc3zOzHwBaAAR+4+7cVi0ykHmSrAQwdCjvvHF4idaCYq4kGp43a0cxw9zvKHJNI/ciWDObOhV120SMwpW4U00y0U2x4DWAPwh3ESgbSdqmZSBpEMc1Ev41/NrPOwJ1lj0iknuhEsTSIlhzWfAn0LVcgInVJyUAaRDHnDP5JdFkpIYlsBdxbiaBE6oaaiaRBFHPO4IrY8ErgU3efWeZ4ROqLagbSIIo5Z/BcJQMRqUtKBtIg8iYDM1tGsnkoZRLg7r5O2aMSqReFNhN9/TV06ADt2lU2HpES5f1LdvdO7r5OhlcnJQJp8wqtGay5JgwaVNlYRFqgmHMGAJjZuoT7DABw9+lljUiknhRzAvnxxysXh0gLFfyXbGb7mtmHwCfAc8A04NEKxSVSH3TOQBpEMdfFXQgMAKa4+8aEO5BfrEhUIiLSqopJBt+6+yJgNTNbzd2fAbavUFwi9UEnhKVBFJMMlpjZ2sA44C4z+zvhfgORtssMDjgg87ROnUpf7tSp8M9/lj6/SJGKOYG8H+E5xb8DDgc6A3+uRFAidaWpKfP4nj1LX2bfqKcX9XoqraSYZDAEGBXddXx7heIRqT/ZkkGuk8vDhsHs2fBnHU9JbSimmWgdYKyZPW9mJ5vZepUKSqSuZEsGuY7qTzwRLrywMvGIlKDgZODu/+vuWwMnAz2B58zsyYpFJlIvsiWDL7+EUaNaNxaREpXS5eJ8YC6wCFi3vOGI1KGrrw53GKebMQMOPhjGjWv9mESKVMxNZyea2bPAU0B34Dh337ZSgYnUjc02C7WAbJ55pvViESlRMSeQNwJOd/eJmSaaWVd3X1yesEQayNix1Y5AJK9iurA+O0+Rp4AdWxaOSAP64INqRyCSVzkf06ROWkQy+eyzakcgklc5k4HujhERqVN6gKuIiKiZSEREiru0dFMz6xgN725mp5pZl1iRPcoenYiItIpiagajgVVmthkwHNgYuDsx0d11lkwknxEjdFey1KRi7jNocveVZnYAcJW7X2Nmb1YqMJGG9KtfVTsCkYyKeriNmR0GHAX8KxrXIdcMZnaLmc03s3eyTDczu9rMpprZW2am+xRERKqgmGRwDLArMNTdPzGzjYH/yzPPbcDAHNP3AfpGryHA9UXEIyIiZVJMr6Xvuvup7j7CzLoCndz9kjzzjANynUvYD7jDg1eALma2fqExidSUffetdgQiJSvmaqJnzWwdM+sGTAJuNbMrW7j+DYAZsc8zo3GZ1j/EzMab2fgFCxa0cLUiFbDlltWOQKRkxTQTdXb3pcAvgFvdvR+wZwvXn+nehIx3Mrv7je7e39379+jRo4WrFRGRuGKSQfuoCedgkieQW2om0Cv2eUNgdpmWLdK6cj3mUqTGFZMM/gyMBT5y99fNbBPgwxaufwwwOLqqaADwubvPaeEyRUSkSMV0YT0KGBX7/DFwYK55zGwEsDvQ3cxmAn8iuhzV3YcBjwCDgKnAl4QrlkTqU6Jm0KcPDBgA99xT1XBEilFwMjCzDYFrgN0I7fovAKe5+8xs87j7YbmW6e5OeKaySOM49liYowqu1JdimoluJTTr9CRc8fPPaJyIQOo5g9VK6ANywAC4667yxSNShGL+Ynu4+63uvjJ63Qbosh6RdO6wcmVx8zQ1wauvwhFHVCYmkTyKSQYLzewIM2sXvY4AFlUqMJG6E68ZPP54YfN4dCX111+XPx6RIhSTDH5NuKx0LjAHOAid8BVpzh3atSusbFNTeP/yy/DeIWd3XyIVU0x3FNPdfV937+Hu67r7/oQb0EQEUmsGhe7UE8ngq6/Ce8eO5Y1JpEAtfdLZGWWJQqSRuMPqqxdWNpEMVqwI76oZSJW0NBnolkuRhJbUDPbZp/zxiBShpckgYz9CIm1SIhm4Q/sCb+FJJIOpU1M/i7SyvH+xZraMzDt9A9Yse0Qi9SqeDLzA46T0nb+SgVRJ3pqBu3dy93UyvDq5ezGPzRRpG9wL36lXIxl8+y2ccQYs0pXhktTSZiIRSYifMyh3MnjsMfjmmzA8ZUrLduT33Qd/+xuceWbpy5CGo2QgUm4tqRlkal564YVwgvncc8PnLbaAbbctPb7EOhPJRQQlA5HyidcMjj22sHkKqRkknuz3YazH+NkteOxH/NyGSETJQKTc3OGEE+CzXI//jjQ1pe6U05PBoEHwiyLu7Zw/HyZPzl0m0YmekoHE6ASwSLn075/63rlz/nnyJYNHH00OF7Lz3mILWLIkd9lEzUBXLkmMagYi5TJoEHz6Key3X/hcyGMwjzsOFi9Ofm5qClf7ZOKefVrCkiX516lmIslAyUCknHr3Tg4Xkgwefhguuih13I47Zi7rnrr8UikZSAZKBiLVtmpV6ud33slczh3mzm35+hLnDNRMJDFKBiL1olxH8jqBLBkoGYhUWyV3ypMmwbXXpo7TCeTCzZ8fbvJrA3Q1kUi1FbpTLiVpbL99eD/55PA+fHjyHojWrBm8/nqokfTr13rrLIfevUP34m2gFqVkIFJtLdnRNDWF+wp+8IPCyp96annWW6ydd279dZZD4jkTbYCaiUSqrdAd5PLlzcdddlnommLChOLXW0vNRO+/D1tuqc7zqkjJQKTaCk0GS5emfr7uOnjppTA8a1bx62pJMij3TvsvfwkJ4V//Ku9ypWBKBiKt5b77MvdZVGqndiefnOxsrtDHbMaV2mTz8MPQvTs8/XRp8+dSyL0ZUhFKBiKt5cADYd99S58/U9JINA+V8uzkUpPB88+H91dfLW3+WrdiBbz2WrWjaHVKBiLVdtNNhZXLlAwWLgzv+WoGmXb8pSaDxHznnJM6vqkJ/vAH+OST0pdZC373O9hll9ReYtsAJQORaujSpfh50u9UjmvXLvXzqFGpO+VMO9tyn0CePBkuvxwOOqj0ZdRCM1GittXGTmbr0lKR1pTYKXfrVlincnG5kkH6jv3gg1N7TW1qSt55nB5LuSRiyNeZXq2rhYRUBaoZiLSGxx8P74kdcilt/LmO5DNN+/zz3NNnzgy9rNaCWmomSqjFmCpIyUCkNfzkJ+F94EA45RS44Ybil5ErGeSqNUDmHduUKdCnT/FxlGMn2dQEM2YkP7/xRni/4orq3/+gmoGIVFz79nDNNbD++sXPm2uHny8ZTJ+efZo7PPdc/vWPHw/TpmVPBvGd6LvvhrLZXHxx6Oph6tTw+b33wvtbb6U+0KeaVDMQkYor5ehz5szs0/Ilg803z35j2nXXwe67w4MP5l7GTjvBxhvnLpOw9da5yz75ZHiP1w4Svv66sHVUimoGIlJ2O+8MZ5zRfHz6ydyWypcMAObNyzw+0StnoecP2soRc7m/pxmceGJ5l1lGFU8GZjbQzD4ws6lmdnaG6Ueb2QIzmxi9MtyiKVKnXn0V/vrX5uPLffRZSDIo184t0/eB8j1BrdpH5pnWv/324RGlLTVsWMuXUSEVTQZm1g64FtgH2Ao4zMy2ylB0pLtvH71urmRMIjWhWskg181niX6OWtsBB5Q+79dfZ+7Arxzi22rSJLi5FXdNK1e2eo+pla4Z7AxMdfeP3f0b4B5gvwqvU6T2VaOZyD13uXvvLX69Q4fmL7NsWe6Y8p2ryOX734dOnYqfb+7c7PdD1MIzon/2M1hjjVZdZaWTwQZA/AzRzGhcugPN7C0zu8/MemVakJkNMbPxZjZ+wYIFlYhVpPWUu2Zw/fXw0EP5y2VKBtliGT48f9fY550X3qdMyX4k+/e/F77OfNPSlXKfxKJF4Wqu//mflq+/UsaObfVVVjoZZNqq6en2n0Afd98WeBK4PdOC3P1Gd+/v7v179OhR5jBFWlm5dzhPPQX775+7jHvma/izHQEfeyz075/5ip+45cthiy3g6KMzL6/YI+z58/OXWbCg9PsRPvoovD/xRO5ybeVEeaTSyWAmED/S3xCYHS/g7ovcPXFIcRNQZ8/FEylBuZuJyi3eI2nv3uHS02y++CK8T55cnnWfcELu6TNnwrrrFtZElUu236AWagbpJkwIz3yooEr/Rb4O9DWzjc1sdeBQYEy8gJnF777ZF3ivwjGJVF81djjFHOkOGZL6uZCb0lpL4n6LUh+Ek9gO6b/BuHFhXKJmUks1g/794dxzK7qKinZU5+4rzewUYCzQDrjF3Seb2Z+B8e4+BjjVzPYFVgKfAUdXMiaRmlDryaDS6ylHLKVuw2zNS3/7W3j/4IPSllvnKt5rqbs/AjySNu782PAfgT9WOg6RmlKNZJDtprNMitlZt2THXsx2WL481AY22qj09UH2ZJAeS2vWDFatgmefhT32aL11pqnxhkuRBlWNcwYTJ1Zmuf37557+1VflWc/xx8NhhyU7tSt3zSB9eZmugsrm229z3++QL7EMHQp77hkuBKgSJQORaqhGzeDGG/OXOe+8sOMqpn+gbH0eQTjavfji5OeFC2HtteHllwtffkKis71sO92VK3P335SQ7V6L9N+kmPsf9tsv9/0O+ZJBomlq7tzm802aVHgcLaBkIFIN1UgGs2fnLzN0aOi4riWPfIzv+K69NnXauHHh6qPEPQnFNMUktlniCWTp2/Css6BXr/yXpiZqBl9+mTp+5crCY0mXr6fVUpucrrkmdIXR0uUUQMlApBpq6dLS9B3MiBHlW3Z6k0yxO7M33gg7/aeeguefD+Muvzy8pyeDR6JTk4nnQueLafLk1HsoKvkchVJ34uk3/SkZiEjFpLfplzNRpTfJFLvDvf768J640zmXQmtb8Zg+/jg5XMiO9v33C1tHulJ34unzVTBhKRmIVEOxO4dSHpNZqGeeSf3ckuYSSD6oBlpeMyil24rddsu9zHhMS5eG5YweXdiO9sc/zl8mk3zfOzE9393bqhmINJhi/6kTXT1UQvzoGEo7uZtNuY5kM9VW0k+2JixZknl8ogYUjylxbuTiiwuLNVene7mUuhP/5JPyLKcASgYi9aCVe7Asm4cfTv2cvjMrNMllqgUk+hjKVSbhscdgrbVCoos3EyWSTFNTYckgvafTWbMK62o639HUHdgAAA83SURBVE48EfuRR6Y+DOnFF4tbTgsoGYhUQ7H/1LXUNUJLpH+PfJ3gFXoeYMaM8NzlbBK9gL78cupO/3e/C++rVhWWDOJNaO6w4YZw6KHJcYMGZZ4v37Lj2yVxJ3Qpy2kBJQORaii2D/6WtuPXilJ3ZvlOat99d2HLMcscQ6E1g4ShQ5MxxbsOz3aJabmSuWoGIg1mjTXCP/YllxRWvlFqBnfcUdp8+WoIZzd7om5w771h3quuSo7LdNNZtu69s4kfvRdy9VWhzUQtXU4LKBmIVFP6P3fiUspGVexDWxI7yXHjSlvfIYekfn7llczdchRbM4j/buVIBoWqYDNRxTuqE5Ec0ncS2ZqDGqVmUKiBA+GBB+CGG8q73GyP9vzww8K74Fi4ED77LPm5Jclg6dLUZ0eUupwyUM1ApJrSj/SUDIKxY8PVP61l5crml3Fmk/6kxfRkcP/9cN99sNVW4f4F9+y/3xFHwF57wahRha27gn8HqhmIVFPin3vttUMHbL175y4ntSc9GRx4YHL4oINg2DA4+ODM8yaeDpd+yWo2aiYSaVCJnfypp8IBB0DfvrnLSe3J10w0ZUrz3++NN+CFF4pfl5qJRBpU4p97tdVyPxcg307gwQfhF78oX1z17rnnQnt8a8iXDNZaq/klp/36wWmnFd97rZqJRBpUPBkUUi6b/fYLfQLdf3954qp3u+/eeuvK99v95S+pzTst6QhQNQORBpXYSSSOEFty0lRNSdWxeHHu6bk660vvUiMf3YEs0qASXRkkrofv0CHsLHbeObVcu3b5l6Vk0PjUTCTSoLbcMvM/ePo4JQMBNROJtDnx5oC99oL2BRy3FbqjiD9GUerL2LH5O/crkZKBSC2K79ivuaZ5MujXD848M/s8ubz5Zstik+o5+uhwI1sFKBmI1KKNNkoO9+jRvJlo/Hi44orUcWomahsq9GwLnTMQqUW33hq6ud5nH+jatXzNRJde2vLYpLrWXLMii1XNQKQWde4Mt9+evNoo/vyDbG3GxdQMXn4Zdt219PikepQMRNqw009PDm+4YXL4o49gwoTilzdgALz0UukPeC+XLl2aj8vWP5MESgYibdjaa2cev8kmsOOOYXibbZpPTz+vkG7kSLjuujC8557FxfSb3xRXHkKNZO+9k5+PPbZ5mWz9M0mgZCAiOR10ELzzDuywQ3JcviuO1lsPTjwRFiyAf/0LzjkHevXKvo5Zs5LD2e59GDgw+/xdukDHjsnPmfrmiT9GUpqr0AlkJQORenHSSfnLbL11aDa67LKwk0+37rqZ5+vePeykhw6F6dOzL79nz+TwxRenNl8lPPpo6IE1k3i/PFddlTkZfOc7cN552WNI2Hbb/GVaqtjaUi6J/pLWX79ly1HNQKSNu/bawk4Sm8F//3ey+SdhxAg46qjS1j1qVPPO37p1S30WcFyfPuH9yCObxxYvk63Xzvj4TCe6DzigtHMlxRo7Fu68s7R5b7stOfzii3D88WG4pQ/tUTIQkRY59NDSe8w86CB45pnCy198Mdx1V7giapddkuPTd/7p8cTPJyQ8/njq5/XWC72zZrrcdv/9C4+xEKutltqslcmBBzbvSwpSa0c//GFhlwfHLw7IRslAREq2xRbFlb/jDnjttczT3n03PD8h4fDDk8OdO4f3jh3hV78KO//4EfKmmybPSXTunDovwGOPpX6+4IJw8nz69HAnNmQ+V/HTn4ZnJp92Wr5v1twll4RnIKebNy+856uNXXtt6vZISE90iUS4ySbZlzVlSvNxP/xh6udKPQ7U3Sv6AgYCHwBTgbMzTO8IjIymvwr0ybfMfv36uYgUaN489+XLS5v3wgvdn38+d5mVK91nzgxP+j3kkMxlRo92X7w4DH/5pfuIEe5NTcnpiScFJ5x/fvh8wQWp6/npT93HjWs+37Jl4fOyZclxidfgwe577918fPo6zzgj8/j33w+f77478/yJ7zV6dHLcj38cvt8xx7g/9FCYPnJkmPaLX4TvkC2WxPCSJe5z5qSOg9J/S3cHxnu2fXW2CeV4Ae2Aj4BNgNWBScBWaWVOAoZFw4cCI/MtV8lApAa9+ab7V1+VNm9iJ5lw331h3OjR+ecD92++SY5bscL95pub79S32675zneLLVKXd9RRzedLLNPd/eGH3d99NwyvtVZqIpoyJTlvonzcmDFh2v77h8/t22dOBhMmuA8bljrvlVcmy6xalXub5FDNZLArMDb2+Y/AH9PKjAV2jYbbAwsBy7VcJQORBvP556k7dPfkTjeXxA4yXstwD0lpr73cX3klOW7OHPe77grld9891GY+/zx1vkcfDdNPOy3/un/zm+aJqF8/95//PHP5r78ONYVp08Lnn/wkzN+/v/vaa7vvtFPu9W2zTfMkVaRcycDC9Mows4OAge5+bPT5SGAXdz8lVuadqMzM6PNHUZmFacsaAgwB6N27d79PP/20YnGLSJ344otw78Pmmxc+zwcfhPMW2dreX3klnPTO93zilSvDc5a7dSt83XFffBHOiVxwQTgpbJZ7nYsXh++a6ebCApnZBHfP+LDtSndUl+mbpWefQsrg7jcCNwL0799f3TOKSLgnoZhEAPlPpg8YUNhy2rcvPRFAiP3yywsv37VreFVIpa8mmgnEb2fcEJidrYyZtQc6A59VOC4REYmpdDJ4HehrZhub2eqEE8Rj0sqMARJ3whwEPO2VbLsSEZFmKtpM5O4rzewUwknidsAt7j7ZzP5MOJExBhgO3GlmUwk1gkMrGZOIiDRX8YfbuPsjwCNp486PDX8N/LLScYiISHa6A1lERJQMREREyUBERFAyEBERqOwdyJViZguAUm9B7k7o8qKW1XqMtR4fKMZyqPX4oPZjrLX4NnL3Hpkm1GUyaAkzG5/tduxaUesx1np8oBjLodbjg9qPsdbji1MzkYiIKBmIiEjbTAY3VjuAAtR6jLUeHyjGcqj1+KD2Y6z1+P6tzZ0zEBGR5tpizUBERNIoGYiISNtKBmY20Mw+MLOpZnZ2lWLoZWbPmNl7ZjbZzE6Lxl9gZrPMbGL0GhSb549RzB+Y2d6tFOc0M3s7imV8NK6bmT1hZh9G712j8WZmV0cxvmVmO1Y4ti1i22mimS01s9OrvQ3N7BYzmx89vS8xruhtZmZHReU/NLOjMq2rzDFebmbvR3E8YGZdovF9zOyr2PYcFpunX/T3MTX6HnkeC9ai+Ir+XSv5v54lxpGx+KaZ2cRofKtvw5Jlex5mo70IXWh/BGwCrA5MAraqQhzrAztGw52AKcBWwAXA7zOU3yqKtSOwcfQd2rVCnNOA7mnjLgPOjobPBi6NhgcBjxKeWjcAeLWVf9e5wEbV3obAj4AdgXdK3WZAN+Dj6L1rNNy1wjHuBbSPhi+NxdgnXi5tOa8RnnFu0ffYp4LxFfW7Vvp/PVOMadP/CpxfrW1Y6qst1Qx2Bqa6+8fu/g1wD7Bfawfh7nPc/Y1oeBnwHrBBjln2A+5x9xXu/gkwlfBdqmE/4PZo+HZg/9j4Ozx4BehiZuu3Ukx7AB+5e6470ltlG7r7OJo/pa/YbbY38IS7f+bui4EngIGVjNHdH3f3ldHHVwhPJMwqinMdd3/Zw17tjtj3Knt8OWT7XSv6v54rxujo/mBgRK5lVHIblqotJYMNgBmxzzPJvROuODPrA+wAvBqNOiWqqt+SaE6genE78LiZTTCzIdG49dx9DoSkBqxb5RghPAwp/o9XS9sQit9m1f47/TXhKDVhYzN708yeM7P/iMZtEMWV0BoxFvO7VnMb/gcwz90/jI2rlW2YU1tKBpna46p2Xa2ZrQ2MBk5396XA9cCmwPbAHEJVE6oX927uviOwD3Cymf0oR9mqxGjhUar7AqOiUbW2DXPJFlPVYjWzc4GVwF3RqDlAb3ffATgDuNvM1qlCjMX+rtX8vQ8j9eCkVrZhXm0pGcwEesU+bwjMrkYgZtaBkAjucvf7Adx9nruvcvcm4CaSzRhVidvdZ0fv84EHonjmJZp/ovf51YyRkKjecPd5Uaw1tQ0jxW6zqsQanaj+GXB41GxB1PyyKBqeQGiH3zyKMd6UVNEYS/hdq7UN2wO/AEYmxtXKNixEW0oGrwN9zWzj6IjyUGBMawcRtSkOB95z9ytj4+Nt7AcAiSsVxgCHmllHM9sY6Es48VTJGL9jZp0Sw4QTjO9EsSSubjkKeCgW4+DoCpkBwOeJppEKSzkKq6VtGFPsNhsL7GVmXaPmkL2icRVjZgOBs4B93f3L2PgeZtYuGt6EsN0+juJcZmYDor/nwbHvVYn4iv1dq/W/vifwvrv/u/mnVrZhQap59rq1X4QrOKYQsvO5VYrh/xGqg28BE6PXIOBO4O1o/Bhg/dg850Yxf0ArXHFAuApjUvSanNhWwHeBp4APo/du0XgDro1ifBvo3woxrgUsAjrHxlV1GxIS0xzgW8KR329K2WaEdvup0euYVohxKqGNPfH3OCwqe2D0+08C3gB+HltOf8JO+SPgH0S9GVQovqJ/10r+r2eKMRp/G3BCWtlW34alvtQdhYiItKlmIhERyULJQERElAxERETJQEREUDIQERGUDERyMrNzLfQu+1bU6+QuFnpIXavasYmUky4tFcnCzHYFrgR2d/cVZtad0AvmS4T7AhZWNUCRMlLNQCS79YGF7r4CINr5HwT0BJ4xs2cAzGwvM3vZzN4ws1FRv1OJZ0JcamavRa/NovG/NLN3zGySmY2rzlcTSaWagUgW0U79BcLdzk8CI939OTObRlQziGoL9xPufv3CzM4COrr7n6NyN7n7UDMbDBzs7j8zs7eBge4+y8y6uPuSqnxBkRjVDESycPflQD9gCLAAGGlmR6cVG0B4yMqLFp5udRThQTsJI2Lvu0bDLwK3mdlxhAexiFRd+2oHIFLL3H0V8CzwbHREn/4YSiM8jOawbItIH3b3E8xsF+CnwEQz296jni1FqkU1A5EsLDxruW9s1PbAp8AywiNLITwZbLfY+YC1zGzz2DyHxN5fjsps6u6vuvv5wEJSu1sWqQrVDESyWxu4xsID4lcSevccQug6+1Ezm+Pu/xk1HY0ws47RfOcReswE6GhmrxIOvBK1h8ujJGOEnkwntcq3EclBJ5BFKiR+ornasYjko2YiERFRzUBERFQzEBERlAxERAQlAxERQclARERQMhAREeD/A+SkKghMwgEiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "steps = step_loss[\"step\"]\n",
    "loss_value = step_loss[\"loss_value\"]\n",
    "steps = list(map(int, steps))\n",
    "loss_value = list(map(float, loss_value))\n",
    "plt.plot(steps, loss_value, color=\"red\")\n",
    "plt.xlabel(\"Steps\")\n",
    "plt.ylabel(\"Loss_value\")\n",
    "plt.title(\"Loss function value change chart\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面可以看出来大致分为三个阶段：\n",
    "\n",
    "阶段一：开始训练loss值在2.2上下浮动，训练收益感觉并不明显。\n",
    "\n",
    "阶段二：训练到某一时刻，loss值减少迅速，训练收益大幅增加。\n",
    "\n",
    "阶段三：loss值收敛到一定小的值后，loss值开始振荡在一个小的区间上无法趋0，再继续增加训练并无明显收益，至此训练结束。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  数据测试验证模型精度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "搭建测试网络的过程主要为：\n",
    "\n",
    "1. 载入模型`.cptk`文件中的参数`param`；\n",
    "2. 将参数`param`载入到神经网络LeNet5中；\n",
    "3. 载入测试数据集；\n",
    "4. 调用函数`model.eval`传入参数测试数据集`ds_eval`，就生成模型`checkpoint_lenet-{epoch}_1875.ckpt`的精度值。\n",
    "\n",
    "> `dataset_sink_mode`表示数据集下沉模式，不支持CPU，所以这里设置成`False`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:18.826530Z",
     "start_time": "2020-10-20T07:22:17.823755Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "============== Starting Testing ==============\n",
      "============== Accuracy:{'Accuracy': 0.9628405448717948} ==============\n"
     ]
    }
   ],
   "source": [
    "# testing relate modules \n",
    "def test_net(network, model, mnist_path):\n",
    "    \"\"\"Define the evaluation method.\"\"\"\n",
    "    print(\"============== Starting Testing ==============\")\n",
    "    # load the saved model for evaluation\n",
    "    param_dict = load_checkpoint(\"./models/ckpt/quick_start/checkpoint_lenet-1_1875.ckpt\")\n",
    "    # load parameter to the network\n",
    "    load_param_into_net(network, param_dict)\n",
    "    # load testing dataset\n",
    "    ds_eval = create_dataset(os.path.join(mnist_path, \"test\"))\n",
    "    acc = model.eval(ds_eval, dataset_sink_mode=False)\n",
    "    print(\"============== Accuracy:{} ==============\".format(acc))\n",
    "\n",
    "test_net(network, model, mnist_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过1875步训练后生成的模型精度超过95%，模型优良。\n",
    "我们可以看一下模型随着训练步数变化，精度随之变化的情况。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`eval_show`将绘制每25个`step`与模型精度值的折线图，其中`steps_eval`存储着模型的step数和对应模型精度值信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:18.957514Z",
     "start_time": "2020-10-20T07:22:18.828056Z"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZxcVZn/8c83CYQACQGSsGQhAZJAcANijKPiuBMU4oJIUNlE1AGUcZnBwUHU0fm5gRuCqFEWARdAI7KOklEYVAKGpSuELISsJAECYQkJSZ7fH+cWqXR6qe509a2q+32/XvWqqrvVU7e673PvOeeeo4jAzMyKq0/eAZiZWb6cCMzMCs6JwMys4JwIzMwKzonAzKzgnAjMzArOicC6RNJoSSGpXxXLnizpjt6IqygkfVDSrdux/k2STurJmDr5vKr/Xiw/TgRNTNIiSRskDWk1fXb2zzk6n8isuyLiFxHx9mqWlXS+pCtbrT8lIi6rTXS9I/vbPTDvOJqJE0HzewSYVn4j6eXAgPzCqQ+NeIbaiDH3pKJ//1pyImh+VwAnVrw/Cbi8cgFJu0m6XNJqSY9K+oKkPtm8vpK+JelxSQuBd7ax7k8lrZC0TNJ/SepbTWCSfi3pMUlPS/qzpEMq5g2Q9O0snqcl3SFpQDbv9ZL+T9JTkpZIOjmbPlPSaRXb2KpoKjuTPEPSPGBeNu272TbWSrpH0hsqlu8r6T8kLZD0TDZ/pKSLJH271Xf5vaSz2/iOl0j6Vqtpv5P06ez1ORXbL0l6T6v475R0oaQngfPb+E5txi/pSOA/gA9IelbSfa33kaQ+2W/9qKRV2d/Abtm8cpHOSZIWZ7//uR38lu3+XpkPtrUdSZMk3ZX9lisk/UDSju39ZpL+nM26L/teH2gvJuuCiPCjSR/AIuCtwFzgYKAvsATYDwhgdLbc5cDvgIHAaOBh4CPZvI8DDwEjgT2A27N1+2Xzfwv8CNgFGAb8HfhYNu9k4I4O4js1+8z+wHeA2RXzLgJmAsOzuP8pW24U8AzpKmcHYE/gVdk6M4HTKrax1edncd+WfY8B2bQPZdvoB3wGeAzYKZv3OeABYDwg4JXZspOA5UCfbLkhwPPAXm18xyOyfa7s/e7AOmDf7P37gX1JJ2UfAJ4D9qmIfyNwVhbfgDa+U0fxnw9c2Sqel/ZRtv/nA/sDuwLXAVdk80Zn++vH2ee+ElgPHNzOb9ne79XhdoDDgclZ/KOBOcDZnfxmARyY9/9XMz1yD8CPGv64WxLBF4D/Bo7M/qn6Zf9Mo7N/2vXAhIr1PgbMzF7/Cfh4xby3Z+v2A/bK1h1QMX8acHv2equDViexDs62u1t2UFwHvLKN5T4PXN/ONl46yLX1+dn239xJHGvKn0tKoFPbWW4O8Lbs9ZnAje0sJ2AxcET2/qPAnzr4/Nnlz8ziX9xqfof7tFX859NxIvgj8C8V88YDL1YclAMYUTH/78DxbXxmR79X1dvJ5p1d+fu29ZvhRNDjDxcNFcMVwAmkg8jlreYNAXYEHq2Y9ijpzA7S2eqSVvPK9iOdla/ILu2fIl0dDOssoKzY5f9lxSJrSUmrHM8QYCdgQRurjmxnerUqvwuSPiNpTlac8RQpEZUr1zv6rMtIZ+Nkz1e0tVCkI9c1bKmnOQH4RcXnn6hUeV/efy+r+Pxt4m2tk/g7sy/b/u7lBF/2WMXr50lXDq119Ht1uB1J4yTdkBURrgW+1kb8He4D235OBAUQEY+SKo2PIl3+V3qcdBa4X8W0UcCy7PUK0gGxcl7ZEtIVwZCIGJw9BkXEIXTuBGAq6YplN9KZI6Qz6MeBF4AD2lhvSTvTIRWr7Fzxfu82lnmpu92sPP3fgeOA3SNiMPB0FkNnn3UlMFXSK0nFbr9tZzmAq4FjJe0HvAa4Nvv8/UhFJmcCe2af/2DF528Vb2tVxN9Z18LL2fZ33wis7GS91jr6vTpzManocWxEDCLVa6jVMu4iucacCIrjI6RL7OcqJ0bEJuBXwFclDcwOTp8mHejI5n1S0ghJuwPnVKy7ArgV+LakQVnl4wGS3lhFPANJSeQJ0sH7axXb3QxMBy6QtG929fBaSf1JZ9NvlXScpH6S9pT0qmzV2cB7Je2s1LzwI1XEsBFYDfSTdB4wqGL+T4CvSBqr5BWS9sxiXArcTboSuDYi1rX3IRHxj+wzfgLcEhFPZbN2IR3kVgNIOoV0RVCtzuJfCYxWVvHfhquBf5U0RtKupN/glxGxsQsxdPZ7VfMd1gLPSjoI+EQV66wk1WtYD3EiKIiIWBARs9qZfRbpbHohcAdwFekfG9IZ6y3AfcC9bHtFcSKpaKlEKp/+DbBPFSFdTiqKWJat+9dW8z9Lqqi9G3gS+DqpcnYx6crmM9n02aQKSIALgQ2kA8VlVBTBtOMW4CZS5fijpLPaymKIC0iJ8FbSweqnbN309jLg5bRTLNTK1aSrn6vKEyKiBHwbuCuL+eXAnVVsq9r4f509PyHp3jbWn57F/mfSFeMLpL+F7mjz96pyvRNIDQB+DPyyinXOBy7LitOO61a0tpVySwYz6yJJR5CunEZnZ8VmDclXBGbdIGkH4FPAT5wErNE5EZh1kaSDgadIRWDfyTkcs+3moiEzs4Kr2RWBpOnZbesPtjNfkr4nab6k+yUdVqtYzMysfbXsxOnnwA/Y9gamsinA2OzxGlJ74td0ttEhQ4bE6NGjeyZCM7OCuOeeex6PiKFtzatZIoiIP6vjbo6nApdnd17+VdJgSftkbdPbNXr0aGbNaq8VpJmZtUXSo+3Ny7OyeDhbt3leypZuDbYi6XRJsyTNWr16da8EZ2ZWFHkmgta3kUM7t5JHxKURMTEiJg4d2uaVjZmZdVOeiWApW/dhM4LU94mZmfWiPBPBDODErPXQZODpzuoHzMys59WssljS1cA/A0MkLQW+SOqymIi4BLiR1GfMfFK3tKfUKhYzM2tfLVsNTetkfgBn1OrzzcysOu5iwsys4Gp5Q5mZmW2PF1+E+fOhpQVKJXjXu+Cwnu+EwYnAzCxv69fDvHnpYF8qbTnwP/wwbMzGCZJg6FAnAjOzhvbCC+ngXj7Qlx/z5sGmTWkZCQ44ACZMgKlT0/OECXDQQbDzzh1vv5ucCMzMetq6dfDQQ1sf7FtaYMEC2JwNX9G3Lxx4YDrIv+996fmQQ2DcOBgwoOPt9zAnAjOrD+vWwcqVsGrVts9r18LIkelM+cAD0/OwYensOS+bN8OyZekMf+7crZ8feQTKXfz36wdjx8IrXgHTpm05wx83DvpXM6xz7TkRmFltRMBTT7V/cK98XrkSnn227e0MHAi77gqPPbbl4App2v77b0kMlUli5Mh0xt0Tnn562wP93LmpOOf557cst8su6eA+aRJ8+MPp7P6QQ1JMO+7YM7HUiBOBWdGsXw833JBao2zatOWxcePW79ua1tkyGzfCE0+kA/yqVanVS2vlSs9hw2CvvdKBs/y69fPQoVvKxdevh0WLUvHK/PnpecGCVOxyww2wYcOWz9hhBxg9euskUU4UY8bATjttHdOGDbBwYdsH/FWrtizXt29af9w4ePOb0/P48el5333zvULZDk4EZkUQAffeCz/7GVx1FaxZs+0yffqkA1350a9f2687mtevXzogHnpo+wf3Pffs3tl6//7poDt+/LbzNm1KxTTl5FCZKO68MxUtlUkwfHhKDLvssqUop1xZCynW8ePh6KO3HOjHj09XIHV+dt8dTgRmzWzVKrjySvj5z+GBB9LB9L3vhZNPhn/6p3TmXD6IN+jZLJDiHzUqPd70pq3nRcDjj29JDJXJYvny1Bzz+OO3HPDHjYPBg/P5HjlxIjBrNi++CDfemM7+//CHVFwzaRJcfHE64BXsIPdSUdTQoTB5ct7R1CUnArNm8cAD6eB/5ZWwejXsvTf867+ms/8JE/KOzuqYE4FZI3vyyVTm//Ofwz33pKKeY45JB/8jj0xl9mad8F+JWaPZtAluvTWd/f/ud6nFy6teBd/9LpxwAgwZkneE1mCcCMwaxdy56eB/xRWpknPPPeETn0hn/696Vd7RWQNzIjCrZ+vWpTL/n/0M7rortY6ZMgW+//3UE2UTNmW03udEYFav1q+Ho46CmTNTZe83vwkf+lCqBDbrQU4EZvUoAk47LSWB6dNT8U8jt/O3uuZEYFaPvvjFVCT0la/AKR7O22rLQ1Wa1Zvp01MCOPVUOPfcvKOxAnAiMKsnt90GH/sYvO1tcMklLg6yXuFEYFYvHngAjj0WDj4YfvObdHOYWS9wIjCrB8uXpxZCu+6a+gcaNCjviKxAXFlslrdnnoF3vjMN4vKXv6RBVcx6kROBWZ42boTjjkvFQjfc4DuELRdOBGZ5iYAzzoCbb4Yf/Sh1EmeWA9cRmOXlG9+ASy+Fc86B00/POxorMCcCszz88pcpARx/PHz1q3lHYwXnRGDW2+64A048Ed7whjSOQB//G1q+/Bdo1pvmzoWpU2H0aLj++jSGsFnOnAjMesvq1elegb594aab0ngCZnXArYbMesO6dWkIyeXL4fbbYf/9847I7CVOBGa1tnlzGkfgb3+Da6+FyZPzjshsK04EZrX2uc/BddfBhRfCe96TdzRm23AdgVkt/eAHcMEFcNZZ8KlP5R2NWZucCMxqZcaMdPCfOjVdDbhLaatTTgRmtTBrFkybBocfDr/4RWopZFanapoIJB0paa6k+ZLOaWP+KEm3S/qHpPslHVXLeMx6xaJF8K53wbBh8Pvfwy675B2RWYdqlggk9QUuAqYAE4Bpkia0WuwLwK8i4lDgeOCHtYrHrFesWZPuFVi/Hm68EfbaK++IzDpVyyuCScD8iFgYERuAa4CprZYJoDwCx27A8hrGY1Zb69fDe98L8+fDb3+bRhozawC1bD46HFhS8X4p8JpWy5wP3CrpLGAX4K1tbUjS6cDpAKNGjerxQM22WwScdhrMnAlXXglvfGPeEZlVrZZXBG01kYhW76cBP4+IEcBRwBWStokpIi6NiIkRMXHo0KE1CNVsO/3pTykBfOlL8MEP5h2NWZfUMhEsBSrH3BvBtkU/HwF+BRARdwE7AUNqGJNZbcyenZ7POCPfOMy6oZaJ4G5grKQxknYkVQbPaLXMYuAtAJIOJiWC1TWMyaw2SqVUMeyO5KwB1SwRRMRG4EzgFmAOqXVQi6QvSzomW+wzwEcl3QdcDZwcEa2Lj8zqX0sLTGjdKM6sMdS0r6GIuBG4sdW08ypel4DX1TIGs5qLSFcEJ52UdyRm3eI7i82217Jl8MwzviKwhuVEYLa9SqX07ERgDcqJwGx7tbSkZycCa1BOBGbbq1SCIUPA97hYg3IiMNtepRIcckjeUZh1mxOB2fYotxhysZA1MCcCs+3x2GPw1FNOBNbQnAjMtocriq0JOBGYbQ83HbUm4ERgtj1KJdhjDw9AYw3NicBse5Qrij0wvTUwJwKz7opwZ3PWFJwIzLpr1Sp48kknAmt4TgRm3VWuKPbNZNbgnAjMussthqxJOBGYdVepBLvtBvvsk3ckZtvFicCsu8oVxW4xZA3OicCsu9zHkDWJThOBpFmSzpC0e28EZNYQVq9OD1cUWxOo5orgeGBf4G5J10h6h+RrYSu4OXPSs68IrAl0mggiYn5EnAuMA64CpgOLJX1J0h61DtCsLrnFkDWRquoIJL0C+DbwTeBa4FhgLfCn2oVmVsdaWmDXXWHEiLwjMdtu/TpbQNI9wFPAT4FzImJ9Nutvkl5Xy+DM6pb7GLIm0mkiAN4fEQvbmhER7+3heMwaQ6kEU6bkHYVZj6imaOg0SYPLbyTtLum/ahiTWX178sk0MpnrB6xJVJMIpkTEU+U3EbEGOKp2IZnVObcYsiZTTSLoK6l/+Y2kAUD/DpY3a24entKaTDV1BFcCf5T0MyCAU4HLahqVWT0rlWDnnWHUqLwjMesRnSaCiPiGpAeAtwACvhIRt9Q8MrN6VW4x1Mc9tFhzqOaKgIi4CbipxrGYNYZSCd7ylryjMOsx1fQ1NFnS3ZKelbRB0iZJa3sjOLO689RTsGyZ6wesqVRzbfsDYBowDxgAnAZ8v5ZBmdUttxiyJlRt0dB8SX0jYhPwM0n/V+O4zOqTh6e0JlRNInhe0o7AbEnfAFYAu9Q2LLM6VSrBgAGw3355R2LWY6opGvpwttyZwHPASOB9tQzKrG6VSnDQQdC3b96RmPWYDq8IJPUFvhoRHwJeAL7UK1GZ1auWFjjiiLyjMOtRHV4RZHUCQ7OioS6TdKSkuZLmSzqnnWWOk1SS1CLpqu58jlmvWLsWlixxRbE1nWrqCBYBd0qaQSoaAiAiLuhopexq4iLgbcBS0ghnMyKiVLHMWODzwOsiYo2kYV3/Cma95KGH0rMriq3JVJMIlmePPsDALmx7EjC/3IW1pGuAqUCpYpmPAhdlHdkREau6sH2z3uVRyaxJVdPFRHfrBYYDSyreLwVe02qZcQCS7gT6AudHxM3d/Dyz2iqVoH9/GDMm70jMelQ1I5TdTupsbisR8ebOVm1jWuvt9APGAv8MjAD+Iullld1eZzGcDpwOMModfVleWlpg/HjoV9XtN2YNo5q/6M9WvN6J1HR0YxXrLSU1NS0bQSpiar3MXyPiReARSXNJieHuyoUi4lLgUoCJEyduk5TMekWpBJMn5x2FWY/r9D6CiLin4nFnRHyabYt42nI3MFbSmKzV0fHAjFbL/BZ4E4CkIaSiojaHxTTL1XPPwaJFrii2plRN0dAeFW/7AIcDe3e2XkRslHQmcAup/H96RLRI+jIwKyJmZPPeLqkEbAI+FxFPdON7mNVWucWQK4qtCVVTNHQPqWxfpCKhR4CPVLPxiLgRuLHVtPMqXgfw6exhVr88Kpk1sWpaDbmJhFmpBDvsAAcckHckZj2umvEIzpA0uOL97pL+pbZhmdWZUgnGjUvJwKzJVNPp3Ecrm3NmN399tHYhmdWhUskVxda0qkkEfSS9dE9A1nVEt/oeMmtI69bBwoWuH7CmVU1l8S3AryRdQqo0/jjgu3+tOB56CCKcCKxpVZMI/p10V+8nSC2HbgV+UsugzOqK+xiyJldNIhgA/DgiLoGXiob6A8/XMjCzulEqpW4lxo7NOxKzmqimjuCPpGRQNgD4n9qEY1aHSqWUBHZ01Zg1p2oSwU4R8Wz5TfZ659qFZFZnSiUXC1lTqyYRPCfpsPIbSYcD62oXklkdeeEFmD/ficCaWjV1BGcDv5ZU7jl0H+ADtQvJrI48/DBs3uxEYE2tmi4m7pZ0EDCe1GrooazbaLPmV24x5JvJrIlVO8LGeGACaTyCQyUREZfXLiyzOlEqQZ8+qXsJsyZVTTfUXySNIDaB1JPoFOAOwInAml+pBAcemIaoNGtS1VQWHwu8BXgsIk4BXkm6j8Cs+bW0uH7Aml41iWBdRGwGNkoaBKwC9q9tWGZ1YMMGmDfPicCaXjV1BLOybqh/TBqk5lng7zWNyqwezJsHmza5otiaXjWthspjD1wi6WZgUETcX9uwzOqA+xiygqi21RAAEbGoRnGY1Z+WFpBg/Pi8IzGrqWrqCMyKqVSC/feHAQM6X9asgTkRmLXHfQxZQbRbNCRpj45WjIgnez4cszrx4oupe4mjj847ErOa66iO4B7SiGRqY17gJqTWzBYsSMnAVwRWAO0mgogY05uBmNWVlpb07ERgBdBpHYGSD0n6z+z9KEmTah+aWY7KTUcPOijfOMx6QTWVxT8EXguckL1/BrioZhGZ1YNSCcaMgV12yTsSs5qr5j6C10TEYZL+ARARayR5zD5rbm4xZAVSzRXBi9mA9QEgaSiwuaZRmeVp40aYO9eJwAqjmkTwPeB6YJikr5K6oP5aTaMyy9PChbB+vROBFUY1fQ39QtI9pK6oBbw7IubUPDKzvLiPISuYam8oWwVcXTnPN5RZ0yongoMPzjcOs15S7Q1lo4A12evBwGLA9xlYcyqVYNQoGDgw70jMekW7dQQRMSYi9gduAY6OiCERsSfwLuC63grQrNd5VDIrmGoqi18dETeW30TETcAbaxeSWY42bYKHHnIisEKp5j6CxyV9AbiSVFT0IeCJmkZllpdFi+CFF5wIrFCquSKYBgwlNSH9LTAsm2bWfMoVxR6e0gqkmuajTwKfygau3xwRz9Y+LLOcuMWQFVA1nc69POte4gGgRdI9kl5WzcYlHSlprqT5ks7pYLljJYWkidWHblYDLS0wfDjstlvekZj1mmqKhn4EfDoi9ouI/YDPAJd2tlLWLcVFwBRgAjBN0jYFr5IGAp8E/taVwM1qwn0MWQFVkwh2iYjby28iYiZQTZeMk4D5EbEwIjYA1wBT21juK8A3gBeq2KZZ7WzeDHPmOBFY4VSTCBZK+k9Jo7PHF4BHqlhvOLCk4v3SbNpLJB0KjIyIG6qO2KxWFi+G5593RbEVTjWJ4FRSq6HrSC2HhgKnVLFee0NcpplSH+BCUlFTxxuSTpc0S9Ks1atXV/HRZt3gPoasoKppNbSGVIbfVUuBkRXvRwDLK94PBF4GzJQEsDcwQ9IxETGrVQyXktVLTJw4MTCrhfLwlG4xZAXTUadzMzpaMSKO6WTbdwNjJY0BlgHHs2WUMyLiaWBIxefNBD7bOgmY9ZpSCfbeG/bYo/NlzZpIR1cEryWV8V9NatHTVlFPuyJio6QzSX0V9QWmR0SLpC8DsyKiw0Rj1utKJdcPWCF1lAj2Bt5Guov4BOAPwNUR0VLtxrM+im5sNe28dpb952q3a9bjIlIiOKWa6i+z5tJR76ObIuLmiDgJmAzMJ5Xnn9Vr0Zn1liVL4NlnXVFshdRhZbGk/sA7SVcFo0nDVroLams+bjFkBdZRZfFlpFY9NwFfiogHey0qs97mRGAF1tEVwYeB54BxwCezJp6QKo0jIgbVODaz3lMqwbBhMGRI58uaNZl2E0FEVHOzmVlzcB9DVmA+2JtFeHhKKzQnArPly2HtWicCKywnAjNXFFvBORGYeXhKKzgnArNSCfbcE4YOzTsSs1w4EZiVK4rVpe60zJqGE4EVW7mPIdcPWIE5EVixrVwJa9Y4EVihORFYsbmi2MyJwAquPCqZrwiswJwIrNhKJRg8OI1MZlZQTgRWbOWKYrcYsgJzIrBi8/CUZk4EVmCrV8Pjj7t+wArPicCKyxXFZoATgRWZO5szA5wIrMhKJRg0CIYPzzsSs1w5EVhxucWQGeBEYEXmPobMACcCK6onnkj9DDkRmDkRWEG5otjsJU4EVkxOBGYvcSKwYiqVYNddYdSovCMxy50TgRVTSwscfLBbDJnhRGBF5RZDZi9xIrDiWbMGVqxwIjDLOBFY8cyZk56dCMwAJwIrIg9PabYVJwIrnpYWGDAA9tsv70jM6oITgRVPqZRaDPXxn78ZOBFYEbnFkNlWnAisWH74Q1i6FCZNyjsSs7pR00Qg6UhJcyXNl3ROG/M/Lakk6X5Jf5TkQlurnd/8Bs48E44+Gj7xibyjMasbNUsEkvoCFwFTgAnANEmtr8f/AUyMiFcAvwG+Uat4rOBmzoQPfhAmT4ZrroF+/fKOyKxu1PKKYBIwPyIWRsQG4BpgauUCEXF7RDyfvf0rMKKG8VhR3X8/TJ0KBxwAN9wAO++cd0RmdaWWiWA4sKTi/dJsWns+AtxUw3isiBYtgiOPhIED4ZZbYI898o7IrO7U8vq4rd68os0FpQ8BE4E3tjP/dOB0gFHuLdKq9fjj8I53wLp1cMcdMHJk3hGZ1aVaXhEsBSr/80YAy1svJOmtwLnAMRGxvq0NRcSlETExIiYOHTq0JsFak3nuOXjnO2HxYvj9730XsVkHapkI7gbGShojaUfgeGBG5QKSDgV+REoCq2oYixXJiy/C+98Ps2bB1VfD61+fd0Rmda1miSAiNgJnArcAc4BfRUSLpC9LOiZb7JvArsCvJc2WNKOdzZlVJwI++lG46Sa4+GJ497vzjsis7tW0DV1E3Ajc2GraeRWv31rLz7cC+vzn4bLL4Pzz4fTT847GrCH4zmJrHt/9Lnz96/Cxj8F553W+vJkBTgTWLK65Bs4+G97zHrjoIg9BadYFTgTW+P7nf+DEE+ENb4CrroK+ffOOyKyhOBFYY7v33nQVMH48zJgBO+2Ud0RmDceJwBrXggUwZUq6W/jmm2Hw4LwjMmtITgTWmFauTHcNb9yYuo4Y3lHvJWbWEXfBaI3nmWfSXcPLl8Of/gQHHZR3RGYNzYnAGsuGDfC+98Hs2fC736Vupc1suzgRWOPYvBlOOQVuuw2mT09XBWa23VxHYI3jc59LzUO/9rWUEMysRzgRWGP41rfgggvgrLPgnG1GPTWz7eBEYPXviivS1cBxx8F3vuO7hs16mBOB1bebb4ZTT4U3vxkuvxz6+E/WrKcVp7J4wQIolVI3xdD2c0fzOnqulSFDYNSoNLLWgAG1/ax6sn49LF0K992Xuo445BC4/nro3z/vyMyaUnESwXXXwb/9W95RdN+QISkhlBPDqFFbv95nn8boY2fjxtT+f8mS9h+rKsYoGjMmjS0waFB+MZs1ueIkgg9/OBUvwJYy5raeO5rX0XNPi4DVq9NQi4sXpwPk4sWwcCHMnAlPP7318n37prtrWyeLyqSx++61LV/fvDkdxMvxtvVYsSItV2ngwBTnyJFw6KFbXo8cCZMmOQmY1VhxEsHee6dHIzn44PbnrV275eBamSgWL4a//x2uvTbdfFVp551Tsthhh56P9fnnYdmyNExkpQEDthzU3/a29DxixNYH+9126/l4zKxqxUkEzWbQoFR23t6g7Js3b7miqEwWy5bBpk09H89OO217gB85MnUI51Y+ZnXNiaBZ9ekDe+2VHq9+dd7RmFkdc1s8M7OCcyIwMys4JwIzs4JzIjAzKzgnAjOzgnMiMDMrOCcCM7OCcyIwMys4Ra170OxhklYDj+YdRytDgMfzDqILGilex1o7jRRvI8UK9RnvfhExtK0ZDZcI6pGkWRExMe84qtVI8TrW2mmkeBspVmi8eF00ZGZWcE4EZmYF50TQMy7NO4AuaqR4HWvtNFK8jRQrNFi8riMwMys4XxGYmRWcE4GZWV5gJGAAAAcySURBVME5EXRC0khJt0uaI6lF0qey6edLWiZpdvY4qmKdz0uaL2mupHfkEPMiSQ9kcc3Kpu0h6TZJ87Ln3bPpkvS9LN77JR3Wi3GOr9h/syWtlXR2Pe1bSdMlrZL0YMW0Lu9LSSdly8+TdFIvxvpNSQ9l8VwvaXA2fbSkdRX7+JKKdQ7P/n7mZ9+nJkPMtRNvl397SUdm0+ZLOqcXY/1lRZyLJM3Opue+b7ssIvzo4AHsAxyWvR4IPAxMAM4HPtvG8hOA+4D+wBhgAdC3l2NeBAxpNe0bwDnZ63OAr2evjwJuAgRMBv6W037uCzwG7FdP+xY4AjgMeLC7+xLYA1iYPe+evd69l2J9O9Ave/31ilhHVy7Xajt/B16bfY+bgCm9uG+79NtnjwXA/sCO2TITeiPWVvO/DZxXL/u2qw9fEXQiIlZExL3Z62eAOcDwDlaZClwTEesj4hFgPjCp9pF2aipwWfb6MuDdFdMvj+SvwGBJ++QQ31uABRHR0V3jvb5vI+LPwJNtxNGVffkO4LaIeDIi1gC3AUf2RqwRcWtEbMze/hUY0dE2sngHRcRdkY5cl7Pl+9U83g6099tPAuZHxMKI2ABcky3ba7FmZ/XHAVd3tI3e3Ldd5UTQBZJGA4cCf8smnZldck8vFw+QksSSitWW0nHiqIUAbpV0j6TTs2l7RcQKSMkNGJZNr4d4AY5n63+ket230PV9WS9xn0o6Cy0bI+kfkv5X0huyacNJ8ZXlEWtXfvt62LdvAFZGxLyKafW6b9vkRFAlSbsC1wJnR8Ra4GLgAOBVwArSpSGkS77WeruN7usi4jBgCnCGpCM6WDb3eCXtCBwD/DqbVM/7tiPtxZd73JLOBTYCv8gmrQBGRcShwKeBqyQNIv9Yu/rb5x0vwDS2Pomp133bLieCKkjagZQEfhER1wFExMqI2BQRm4Efs6WIYikwsmL1EcDy3ow3IpZnz6uA67PYVpaLfLLnVdniucdLSlj3RsRKqO99m+nqvsw17qxy+l3AB7MiCbIiliey1/eQytnHZbFWFh/1aqzd+O3z3rf9gPcCvyxPq9d92xEngk5k5X8/BeZExAUV0yvL0d8DlFsTzACOl9Rf0hhgLKmCqLfi3UXSwPJrUmXhg1lc5dYqJwG/q4j3xKzFy2Tg6XKxRy/a6oyqXvdtha7uy1uAt0vaPSvqeHs2reYkHQn8O3BMRDxfMX2opL7Z6/1J+3JhFu8zkiZnf/snVny/3oi3q7/93cBYSWOyK8vjs2V7y1uBhyLipSKfet23Hcq7trreH8DrSZdv9wOzs8dRwBXAA9n0GcA+FeucSzoLmEsvtwogtZ64L3u0AOdm0/cE/gjMy573yKYLuCiL9wFgYi/HuzPwBLBbxbS62bekBLUCeJF0RveR7uxLUvn8/OxxSi/GOp9Uhl7+270kW/Z92d/HfcC9wNEV25lIOgAvAH5A1gNBL8Xb5d8++398OJt3bm/Fmk3/OfDxVsvmvm+7+nAXE2ZmBeeiITOzgnMiMDMrOCcCM7OCcyIwMys4JwIzs4JzIrBCUOrVdOe84+hI1mvlg50vadaznAisKM4m3bPQtLK7XM26zInAmkp2Z/UfJN0n6UFJH5D0SWBf4HZJt2fLvV3SXZLulfTrrC+p8lgOX5f09+xxYBufcX7WIdpMSQuz7W9zRi/ps5LOz17PlHShpD8rjW3xaknXKY1P8F8Vm+8n6bKs07XflK9ilPqx/9+sI8FbKrq4mCnpa5L+F/hUTXaqNT0nAms2RwLLI+KVEfEy4OaI+B6pT5c3RcSbJA0BvgC8NVLnfLNInYOVrY2ISaQ7P7/TzuccROpeehLwxaw/qs5siIgjgEtIXQucAbwMOFnSntky44FLI+IVwFrgX7Jtfx84NiIOB6YDX63Y7uCIeGNEfBuzbvClpDWbB4BvSfo6cENE/KWNZSaTBjq5M3X5wo7AXRXzr654vrCdz/lDRKwH1ktaBexVRWzlPnAeAFoi69NJ0kJSx2lPAUsi4s5suSuBTwI3kxLGbVm8fUndHZT9ErPt4ERgTSUiHpZ0OKn/mf+WdGtEfLnVYiINFDOtvc2087rS+orXm0j/SxvZ+ip7p3bW2dxq/c1s+V9s/XnlrpZbIuK17cTyXDvTzarioiFrKpL2BZ6PiCuBb5GGFwR4hjTUKKSRul5XLv+XtLOkcRWb+UDFc+WVQmdWAsMk7SmpP6nr564aJal8wJ8G3EHqZG1oebqkHSQd0o1tm7XJVwTWbF4OfFPSZlJPkZ/Ipl8K3CRpRVZPcDJwdXbAhlRn8HD2ur+kv5FOlNq7athGRLwo6cukEeweAR7qRvxzgJMk/YjUu+nFEbFB0rHA9yTtRvq//Q6ph0uz7ebeR80qSFpE6j768bxjMestLhoyMys4XxGYmRWcrwjMzArOicDMrOCcCMzMCs6JwMys4JwIzMwK7v8DlSBTXuPILioAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def eval_show(steps_eval):\n",
    "    plt.xlabel(\"step number\")\n",
    "    plt.ylabel(\"Model accuracy\")\n",
    "    plt.title(\"Model accuracy variation chart\")\n",
    "    plt.plot(steps_eval[\"step\"], steps_eval[\"acc\"], \"red\")\n",
    "    plt.show()\n",
    "\n",
    "eval_show(steps_eval)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从图中可以看出训练得到的模型精度变化分为三个阶段：1、缓慢上升，2、迅速上升，3、缓慢上升趋近于不到1的某个值时附近振荡，说明随着训练数据的增加，会对模型精度有着正相关的影响，但是随着精度到达一定程度，训练收益会降低。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型预测应用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们尝试使用生成的模型应用到分类预测单个或者单组图片数据上，具体步骤如下："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 需要将要测试的数据转换成适应LeNet5的数据类型。\n",
    "2. 提取出`image`的数据。\n",
    "3. 使用函数`model.predict`预测`image`对应的数字。需要说明的是`predict`返回的是`image`对应0-9的概率值。\n",
    "4. 调用`plot_pie`将预测的各数字的概率显示出来。负概率的数字会被去掉。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "载入要测试的数据集并调用`create_dataset`转换成符合格式要求的数据集，并选取其中一组32张图片进行预测。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:19.982078Z",
     "start_time": "2020-10-20T07:22:18.958520Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Row 1, column 7 is incorrectly identified as 6, the correct value should be 5 \n",
      "\n",
      "[4 3 2 6 0 7 6 7 8 4 8 7 7 1 4 2 6 0 6 3 0 0 1 9 2 1 4 7 4 0 6 0] <--Predicted figures\n",
      "[4 3 2 6 0 7 5 7 8 4 8 7 7 1 4 2 6 0 6 3 0 0 1 9 2 1 4 7 4 0 6 0] <--The right number\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADsCAYAAADXaXXTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOyde3wU1d3/32d3Qy4EEhKuAUIIkBALiphCQSjRPoA3LFZBEaWiFEWpfQS0+uCvUkuqVoSnykWRWxXUihXrtWBboYAKRi7KQyRACPdLEhIgV5Ld8/tjdjd7TTbJzuwGzvv1yiu7M2dmPnNm5rtnvuf7PUdIKVEoFAqFMZhCLUChUCguJ5TRVSgUCgNRRlehUCgMRBldhUKhMBBldBUKhcJAlNFVKBQKA1FGV6FQKAwkrIyuEPQRgiohWB1qLf4QgtVCcFIIzgtBnhBMCbUmXwhBpBAsF4LDQnBBCHYKwY2h1lUfQnCXEOQKQbkQHBSC4aHW5IkQJAjBOrvGw0Jwd6g1+UMIyjz+rELwSqh1+UWIuxAiFyHKEeIgQoTd9Yfm16tFB0EWKalt4uaLgG+Cqac+mqj1OeABKakWgr7ARiHYKSXf6iDRSRO0WoCjwAjgCHAT8K4Q9JeSAh0kOmlKvQrBSOAF4E5gO9BFD20ex2zK9V8EXAQ6AQOAT4Rgt5T8X9AFutAUrVIS67J9a+A0sDbY2rwQwoKUjatXIQy//tphQ1CvUsqA/kAWgHwK5F6QJSBXgowCmQXyGMjfgjwF8k17+VtA7gJZCvJLkFc2sP+7QL4Lcg7I1YHqCoVWl+OkgzwJcny4a7Vv+x3I28NRq339A8257nrrBNka5EWQaS7L3gT5fLhp9XGcX4LMBymaXLdQIOEpCXsllEhYKSFKQpaEYxJ+K+GUhDft5W+RsEtCqYQvJfjXqq0PyvUP93pt7EnsAdkdZALIrSDn2k+iFuQLICNBRoMcCPIMyMEgzXZhBSAj7ftaDHKxy77bgsyz7ztYRlcXrS7LKkBKkDtAxoarVpfjdAJZBbJvuGm1r78I8kmQB+wPxUKQ0WGm82qQlR7HmgXyo3CrUx/H+TfIOc15ruxGd4+E7hISJGyVMNdudGslvCAhUkK0hIESzkgYLMEs4Zf27SPt+1osYbH9s1nCRQlPSjhgN+ALJTTp+od7vTb2JB5y+X4TyIP2k7gIMspl3RKQf/DYfh/IEX72/WeQv7V/DpbR1UWrSxkzyGEgnwYZEeZaI0D+E+Rr4VivIJNASpA5ILuAbG9/SLLDTOdwkKc8lv0K5MZwq1OPMskgrSB7Nuf6243mQy7fb5Jw0G50L0qIclm3RMIfPLbfJ8FbKyRJ7QbIkdBFQnu7QW/S9Q/3em1sR9pRl8+HgST750IpqXJZ1wOYKQSljj+gu0t5J0IwAPgvYEEjtRiu1RUpsUrJFqAbMC1ctQqBCXgTzQ85vZk69dJaaf//ipSclJIiYD6aHzqcdJYBbT2WtQUuNEOnXlpdmQRskZJDzdRZr1ak9NKKEKXOvwCuP1KeRMpgXP96tYayXhvbkdbd5XMycML+WXqUOwpkS0l2APvMAlKAI0IAEAuYheAKKRnYSH16a/WFBejVxG0d6KJVCASwHK3T5yYpqWmmTtBBq5SUCMExH/toDnrUaR5gEYI+UrLfvuwqaHYnmt736iTg+SZq86RRWpGyYa1SliBEsK8/hGu9NrK5/j3Ibmg+ks0g/2hvrh/zKJsJ8iiaj0SgdUDcDLKNj/3GgOzs8jcP5HsgOzTz1UIPrR3ROvxi0dwLo0GWg/x5uGm1l38V5Nc0w+dsoNZnQX5jr+N29n3/IQx1vgPybXu5a0GeA/mjcKxT+zZD7feo3zIB/2nuhe8ldLP7dDdL+KN0dKS5l82UcNTu0xUSWku4WYJvHfCshG8kdJTQzr7vJl3/cK/Xxp6EozewFORf0Aym10nYy99gf4hK0Xr41zoE2o3Bq36OM4fgRi8ETSvIDiA32cudt1/UX4Wp1h4gJVrnWZnL38Rw02r/HoHWYVGK1qv8Mi5+tzDSmQDyA/sDdwTk3eF4/V3Kv4a9h77Zf+7RC6US/iIhxqfR1crfYDekpRJOSljrNLrwqoRXXcpGSK1zrVRqERAvu/mIL6F6FdoOGkYICoApUvLPgDYIIUqrPrQUrS1FJ7QsrQhRAExByrDXGs71GlYZaQqFQnGpo4yuQqFQGEjA7gWFQqFQNB/V0lUoFAoDUUZXoVAoDKTe5IiRpnFh5Xv43LZW+FuntDYdf1pbik5QWpvDpaC1pegE1dJVKBQKQwn6eLoKBcDF0ZnImUXO79EzorDt+SGEihSK8EAZXUVQqRw7CIDIX5/k84yPnMtHJk5Wr1UKL+S1A9h/v28z1H5rBAAJK74yUpLuKKPrgzPTh1LV3nt5VBF0XPil8YJcMF2VQcFt7Xyu67BLGwA/+oPtRkpyUjl2EEwrBHAa3BJrBZmbHiH9xDmsIVGlCEesWdpYVken1XJo+DKfZSb/SJutZ2/tEOLfuHQMr2p8KBQKhYHo2tI1d+hA4S293Za131GKbXeunodtMiIyktJxV/PUo2sYH3vOa/27ZXHML51A/NqdyOrqECiEooHx5E5d7HPdyNwxAFyszKTV+hwjZQFwdIyNQ/3XOb/n1ZQzbucU0qfnYy31rk/F5Ym8dgBHp2lvZfuGv+G33MrkzQBk3ZMI/ou1OHQ1ujVXdOOb7CVuyzJee5jk3XoetfGYO3Sg5opuXGwbwdvZ8+gZEeuz3PjYc/w4ex73nZtBzMZcbBeaO3Z1gPr6pFKTFAdAWQ+/kSjOV/qsmWNhvSHSnJj69aVtYrnbsnfPXUPnsbktzq0gMvthbR3RYLmIvcewFhYaoOjSwdSvLycer2bfoLfdllfLGl4u6Uu1zbveDx9pT5q+c6kaim5GV0RGcrGtewUeqy3DdNGjnMWCKTHB+d1Wes6QVqSpTRtETDQAJ+/ozc7Zjtajb4ProGdELJteW8rICZMxbdqps0owx8ex75k4Dl6/UvdjNYfK+VXs7veB83uZrYrDlYlAuf+Nwg0hMLdvz3WrvuLxhIMNFr86+2E6ryzHVlFhgLiWjzkxAdvLF9jd9xO35dWyhs8q2vHFdak+f8TSMP6tzRwfB5GR3itsEmtRkTZqahPRzeiWjruat7Pn4WrExrzwBMkrd2FzKWcb3I8Fa+pelyc/PYO41V/rJcvJvuev4JObtRmC2phsNGRsQ8XxVUnkXLMIiAm1lEZxR97tmMZV0pKMrrl9e6Z++RU3xpQADbd0P/rtnxjd9gm6PRfaztWWQtdPq1nQ9VMgym35yyV9NYNbVOR7wxBwfFUS7wxY7rV8X01Hlg4d0qw3HN2Mrs0i3F7TBz47jS5/24/Vo1UgLYKMVjFu2xmBjLK6HTdcaRdTSTuzt85DNWXcN30GE//0MVPjTvjY0jjOfpzGmvSVQGuu2j4BgKSnJdbi4yHVFSgVvxjM6DmbiDTlc2NMCZGiYYML0M0Si61V849/ZvpQAG6f+m+35Vvuz0Tm7KEgewj3jvmi0fs9XJnI8ZsisRafbb7IZmBOTKDrp9VkJ20g1tQawHmfdFwQhbm8Blm4J5QSgTqdPaKLGR+3jLSI1j5KnQFT82xU0I1u6aQhAAz/9Ta35Ym5VV6/DhdHZ9L1mQNuy6QxNrdBUtc9SK++J9xiTY0mb1kmK3ut8Fr+blkcL8ybhvmhIrJi9gO+bg7j+Hnyd84b9Hyx9r/znsBfCc3xcRxflUTS09KwBIoTs4aSOEr7sRqU+A1Pt3ccNzCDG0yqErX/dRo07nm5M8fKUvhdj3eZ2Ka40fvNvVjBTMvYYEhsHhYLMzp9TEdz3X3qvE825QR9YrTG4kjkibLUsKDrp8SaotDzmQq60T2fqlnNl7rsaLBsWVIEq1M2Or/3/HAqfXeUurkf9KL7RyYyjj7sv0B7K8PaN+zX05MHBm0hK9q9Nl4824u3Xh1Nl78f4KX/+cDPr7ExmNq0Yd/zV/CHNguBCH76/W10/yjwKERHYLwp0krONYvImvsAnV4cgNi6SzfNBdlDuNi5hvsyN/JMh70Nlp98RIsV/fof/anqXMuhW5e6FwhGI8HPPlyfjZaI6/XtbK5b3tj7RA/O3j+Eomu1eVp7p5x2aVxF+d1mQ0UEjy2fRfKF5t2fQTW6lWMH0WHISbdlvoLjHYHR529w9/f1frvGsHCy6A+2k/yB+zJTTAxHfjMAWyRMy/qnz86UcAj2r2oPBx7rRaLZvY3w+8IrACjekESSAb29IiaaT25e4HTTFH7VheQP6vdvVo4dROEA7bazZZS5BMbHsHvQ26Q/OonuEQMxb2z4R7sxOK7tK3e+zqgY70mRt1fXcOe/pwHw1+uXMCjSvcXbNl/Sc00RvWInkzNikU+XT1NxJLX0/GyK23JfOtZcSOTpLbd57eO+zC8D+hExAt+JD+715bgHGDDUkKQj12cb4Cc3fO8MSQuU7RW96Pbcl81uFKrkCIVCoTCQoLV0L47O9Mq39xccf3yE1oTfN1yLWiizVXFH3u1YiisNcS34wtyhAyfH92H9tD/RzeI7ksHoYP/l24cx7Lo8NxfD4wkHedyZHOHuWliVo3XIpM0LUW96A845xz2SW4+ffN/wN8jY+zDJG4Mnq6Fru7HSxOQvppE2RfND37lsGiuvW0FWtM0tQN/6RgHpMztyahu0c7wuB8Eh6UjbTvN487pz2TR6JLv36B/f0YW0J71TYlctH8ozN4ZHS9fz+fbkP/3XQf+670YkHYk2sbz34Lx6O88ddqiqNoI5vT70cu0Fi6AZXTmzyMvgTvx+sldwvLlPKlVdat22PVprwzShButp40ahck04ACjMiLLH6voPHdtY0YdOL0ZiKzMmDCptSg6Tl93PA4O2MChGc3X4ejUGzd/U6pRxnUCmNm0oH5RClAjsxhSZ/ej6zAE3P2VeTTnvnruGSFMNj7b7gUgRwbtlcUQ1vs+oXmqu6OZ1bTdURLC9oheg/bg5DC7U1ftff+b9em8krpocpPpwG/lKTDljLWf+6VFQW+tVXi9M/fpSmxjt9Xy78m5ZHCkRRW71Oj72HP3nzmfm52Oxnj5jhFQ3HPfh4cpETOMqiSw+zpwNt7LRJe48r6acvx+5kgTymn28oBhdc2ICURZ3YzB130QSbslzK4PFwr5n4jh0fV1nRLWsYV9NR7AZ14fZ1ISDqXEnGL5mMTMHG3dzpE3JYTNRvP3UrwB8ttaO1Zbx2PInSDEwXrR2YG82vbYUhyHzlfgC+Ew4KLFqYYPjdk6hyx37MaWl8vPPviMtIoJ5f7ybjquCdx7+knQeW14XX+sr+D5tSg53LZrujOUuqYimi8UCifGYRaj72+vwl3CwtOQajgwux6g4aYeOz/u+47fMoZoy5j/7EKevq+XQjb4HudENm2RfTUdSI0ootFZzwVbnWb1r1xQ6j81Fq6vygOxZcwiK0fUX9OxZZkanj+29mHVN/FAERrfEhIPkP2s9pmPOP+GSPafhK+nEaPxp8JVwkLnpEQDSp+dTO7gfL7+1iF6WaF10BZqk44v0J/cy81kt5Kpr9QndtTaFQJ69cNFx3/QZxG/Yyenr+vstoxfWoiKWDh3C/i/yeWvJaLq8Vxeq2rX6hNvbuN512iyj6yvo2cHS9DW8+901zu9T2211i9NzUG2LMDR//ezHaaztv4x2PrS4krH0YVoNKGG3R454qDh1/wAAnnp0jde6iHIZ8lRUXxpEZj+uW/WVW8JBzw+nkvGS9gNrLT2HtAhdwt4cCQdPPbrGayyNQOvLduEC2MfXqPjFYO7N/iikIXq+6BFdbI8rreOegiyKHu0G6JtwIDL7MWyF9pYwtd1WLxtwT0EWp5/q6fwesyMXW4gGikJKrIWF/Ou+IXQ5vL9em+Nap67JPsFq1DSvpesj6NlBWkRrj2Bv7zIzTw5ke/aPiWGb1zq9cA3k9+RYbRljXngCS4UkddMJ9nb3MahuCDgxayiT7tNGsPE1+tnwX2/j27tTvJaXVETT9b4ThnT6+UpqsbaOsLsUNIObuu5B+i49j3V/PuCdHJOx9GFS/3OCYHghHQkHnvXV1GNUxZlCnvkXKMfK4onM0T/Dy9o6wuUZ936mjpXFE+kyPkko38QcyJw9fkM9HUk64+OW4TifpiT7NERIBzH/tjiZmPeDa3BPzBpKWUadczE2txVJDfTmv3i2FyveG42pGu21s6KCI7OGcl/mxqBqawonZg3ljns31jsAy0tddoCPZJQSawWZCx/BVm2mz4paXZMO4u85RmXJoHoHUE/MMWHbnes2u8TqlI2csZYzbM0s+iw/Qu3RY8ER5PEj0NRjOLTG3+O9TfrmSXTfVNUsmU3FMzGlpXDCx3MVrKSDZhMZyTsDljsbZXolcTTL6MqKSm7+5DHeuWmhz17e7dU1TNg61SuYXM8g/sRRJ/jepdcxq/tYmFe3/s2PrmN552Fu28TmtiLZbphtaK+mk+5b72XotlfXcNenj5FeYUxojj8dgdLOHOPsLEyPnYRp5FDnurb5slmj8UecOEevf9clCnye8REZAx72Sjhx5fwN5ZT1GOpMoPk84yN7nc4gPXsPtToOlVlsFfRecJDaRnSA+poJw8FV2yfQ42UTYmtwkzgCxTMxBYxPjmmIm5O+563po+m8YpczMeH+O9zv5zUXEnn2vfGkBCHpoDlYundj/yPd3RKOAkn2aQoqOUKhUCgMpFktXduFC/R5ZJvPIG7QBh/OmH3ELZj8xbO9eO/NLIAGX/uDwTWJR/jP5CHO773+94BfJ3p9M0c4A+gf2abrL7LrbBv+ZrBoCvuGvwHD675n7RnbrNH4rfvzvRIFbBllnLXXdeyJGixV7t4zTw261qlLVFdeTTl37ZpC1+rAfLIVvxhMVZyJ+HuOebVwHQH02uA8oXkdNnfowMk7etuHJK0j5MkxHjyecJAej67hOTnRK9RxzQXN6T5n3XhSZ4d+/rOLqR3Im7QEhy935smBtM3XJzQwKD5dX0HcAH3bFFOeleEWQL9i7Wjnq7wRvNRlB2TXvQIOjJxGYm43n2XrmzlizsFb/Z5nMPE124YnZ6zlLC25pt4yHSPOe3X8uCYEBGU0/tpa5p8eSXbSBjqaW7sZ1XsKstj7Rka9my87/VPSX63SfZQpX0ktnskxrtyb/ZFX3XkG0Idy2Ep/yR5GJscEyvjYc4z3ofXZ98YDhIXB9cXmVwbTTqfJMHXtSAs4gN5AdvyufoPmenOcsZZTbNV6ZEoqoumso66GcCQUnLLC/NOj7IHv/pHX/oTha9zjeR9bPqvehIDGYi0+y5HBMHvbKBZ0/Zdb6NLqlI3wu431br86ZSNzV/Rl85X6xphOjTtB1luL+O8bJ0NxKUDAyTGuiRyuAfThRCiSYwBErSSvppxelmjMIjBPZai0hhOGRi+EQxB/Yxi2Zha9F2hOf88AaqNxJhTMPG5P7az/wTdt28PMwe5jqSZf0Kfuj98UyR1rb+cfHllR4UQvSzT/+9lKrPbYNs8kHX+4JnKE61xvoXquTNv28N83TuZ/P1sZcPxyS7MBemCo0TUiiD96RhQjEydzYIL2quU1/mmAZCx9WAsvCkEuuCduCQUB6pG1tYalKluLz2J6tC8jEyc3eltzeQ16B/EDmIUpYMMw8NlpJOZqoWDpJzSferjMZuxI0nAlVMkxsrYWW14+0+9+hDuWbfAbx+yY5aTV+Rq67PWePSbU+KpTPdHN6OoZ+F4ftj0/YAL6lmr+xFTrg+Tf9lqD271bFseLz93t/J666UTwYkabgWdCQbjiqPfGEi6jGLhe/84f13W2hkvr1jEjy7DfbPMybqGcbUXWavHfb84ew+txvu8Ac40kfoM2gli41Kcrnokvetsq3YyuY1YIXQLfA8AxGHr60gwyCuuZIcJOVBF0XFnnZzJubCZ3WuUXkrG0Tm/6uhLDBna/lOhqT1rIEA1fe3C//uFoGBozI0soiHl/W73OmnD5cQ2Ebl9UUptfoNv+dTO67fIq6fnZFESVWffA9/qw7c4leXdIDt0kao8eI3lO3Y/T5ez7ag6OmSeCOS5vOHLV9gl0yqsMtYwWiecMNkbNCqOSIxQKhcJAdGvpiq27SNuqfVatNYWimfh5P++4IAqxdafvlYp68Zzh4pRViw7SuwM6pAPeKBQKRahwzFAyt6gvAIcrEw2ZaUMZXYVCcVnimIF480JHco4xiS9CypbUr6hQKBQtG9WRplAoFAaijK5CoVAYiDK6CoVCYSDK6CoUCoWBKKOrUCgUBhIWRlcIUoTgUyEoEYJTQrBQiPAOZxOCPkJQJQSrQ63FHy2pXoWgzOPPKgSvhFqXJy1FJ4AQTBeCHCGoFoJVodYTCOH+XAlBpBAsF4LDQnBBCHYKwY2N2UfQjW4TH+rFwBmgCzAAGAEENlJJM2imAVoEfBMsLQ1xqderlMQ6/oBOQCWwNujiXGgpOqHJ1/8EMBdYEWQ59XKJP1cW4CjasxQH/D/gXSFICXQHARtdISgQgqeEYK+95bRSCKKEIEsIjgnBb4XgFLDSXv4WIdglBKVC8KUQXFnP7nsC70pJlZScAv4B/ChQbQZrRQjuAkqBfzVVo0FaW1S9unAH2o/F5ktZp95apeR9KfkAKG6qPqO02suH/XMlJeVSMkdKCqTEJiUfA4eA+ufPct+JDOgPZAHIPSC7g0wAuRXkXJBZIGtBvgAyEmQ0yIEgz4AcDNIM8pf27SPt+1oMcrHLvh8C+QbIGJBd7ce5LVBtBmttCzLPvu85IFc3Vaeq1zqtHsf5N8g5l7pOo7Ta97eqOTrVc+X3HugEsgpk34C1NfIkHnL5fhPIg/aTuAgyymXdEpB/8Nh+H8gRfvadAfJbe2VIkKtAimZWuF5a/wzyt/bPwbo5Lvt6dSmTDNIKsuelrtNArcE0upf9c+VSJgLkP0G+1hhtjfXpHnX5fBhIsn8ulJIql3U9gJn2pnqpEJQC3V3KOxECE7AeeB9t/uP2QDvghUZqM0LrAOC/gAXN1GaE1hZTrx5MArZIyaHLRKcRWoPJZf1cuWg2AW8CF4HpjRHVWCdyd5fPyWiOesBr4LmjQLaUZAewzwT7fhdKSTVQLQQr0ToAnmikPr21ZgEpwBGhDeQfC5iF4AopGRhmWltSvboyCXi+idpcaSk6QX+tweRyf64QAgEsR+tIvUlKahqlqpHN9e9BdkPzkWwG+Ud7c/2YR9lMkEfRfCQCZGuQN4Ns42ff+SCfBGkBGQ9yHcg1zXy1CLpWNN9oZ5e/eSDfA9kh3LS2pHp12WYoyPL6ylxKOg24/haQUSCfA/mm/bMl3LS2wOfqVZBfg4xtkrZGnsRTIPeCLAX5F3tleZ2EvfwNIL+xlz0Jcq3jJOyiX3UpOwDkRpAlIIvsZTs2s8J10eqx3RyC43tS9aotew3km82pz5ak04DrPwek9PibE45aW8pzBbKHvR6rQJa5/E0MVFvAQzsKQQEwRUr+GdAGIURp1YeWorWl6ASlVS/CWWtYZKQpFArF5YIyugqFQmEgauYIhUKhMBDV0lUoFAoDUUZXoVAoDKTe5IiRpnFh5Xv43LZW+FuntDYdf1pbik5QWpvDpaC1pegE1dJVKBQKQ1FGV6FQKAwkLGcRUCgUvjFdlUHBbe0wVUPyn3dhq6gItSRFI1EtXYVCoTAQw1q6Fb8YTFWcbxtvrpHEr92JrK42Sk7AmK7KAKBoYDwQeq2W1BTOjOhSb5nYEzW0Wp9jkCKFkRQNjCd36mJyL1Ywc8VYuARauuYOHSi8pTcAHT4+gLWwMMSK9CWoRtfcJ5WapDif6+7N/oipcSd8rsu9WMHMz8diPX0mmHKajblPKrnT2gBw6NYlQGi0utZr/nXR5E5dXG/5ewqyKCruh8zZY4Q8NyzduwFwMbUDAKJWYtq2B1lba7iWSxERVn30waHmim58k609XyPzJ2PaZKzRNXfoQM0V3Rq1jWXHAWwXLjTpeEE1urlPJnDoxmXB3KVumNq0QcRE1y2orcVafNatjHY+Sw1W5o45Po59z8Rx8PqVAW+zOmUjc1f0ZfOVUToq803+/ckA5D6o/TDkXqxg5uDw+kE1x8dBZGSD5eSFsrD1mZqFhMR4RPHZsPpBC+S5ckVERnKxbYQByvxTdHNvtv9xSaO2GTlhMqZNO5t0vMu2I23f81fwyc11A9XPPz2SI4NDKMgPx1clkXPNIiAm1FIuGY6vSuKdAcsbLHfHa7Po9tyXBihqPL0s0fzvZyuZfvcjiK27Qi3HSWOfq9JxV/N29jy0ccsvD4JmdPOWZfLX65cAof3VCoS8ZZn89WcLyWhVZ8h6RBdzBONbhg3RLqaSdub6DW7G0odpNaCE3YPe9lsmb1kmDwza4vy+fPsw0qbo7/dNjYjgun8d5F/3DQmJu6PiF4MZPWeT27LxcctIi2jd4LYLHnidX8dOASBl9le66GsqZmEiLaI10qJNYxAuyCir23OVnbSB2dtGcfymSJ8tXptF0DPi8jG4EESj2yO5iEGR3gb3WG0ZY154AkuF5oyKv+cYn2d8FKzDNgl/Wl3Jf34Ic4e9a5CixvFuWRwvPne383vqphPUfB5P+qOTANg3/A2vbXokF/F0+x+c3/+Z3Fd/oUCkiODxhINsaP1TZ6hM6aQhAETfc5LDR9rravyr4kxu563RsMEFGBVTw+/u0O6BOebxpD4ZesMrw8nCeuDrmelobs2MTp8z0zLWbbnjHhj+622G6fNHh/+cJGPpw259JanrHqRX3xO62Cpd3QsbKiJ4bPkTJK/U4glPzBrKsPYHfZSZRfKF8HlFAug68CQT2xS7LTNaa0H2EH7Xw/0mfvFsL956dTQdV9a99tYCIh+6R2jTSWXkPkxUEXRu8z37nr8CGWVlbo91hmj21+w681gV5ydlApD1I63FuzJ5M3M792WzDm8YlWMHAdqPvD8mHxkOwNf/6O+2/M/3vM6oGG3aK8c98PrAk0HXeKnRmGfmfKp2o7zUZQcAZ6zlDFsziz75RzHaQ12bX0Dq8loyeLhuYXurl62C4OjU3adri4SCJwYAcP8d60VaHtcAACAASURBVHk8wf1Etlf0ottzX2LTW4idM9OHcnfS+iZta7TWe8d84XUTB0pVezg4ux9f3zqPjua6lt3vC68AoHhDEkkUBEOmk8qxg+gwpM44ba+uYcLWqeSMWFSv60MPCgdot3auR0vlqu0TOF+s1Uf7rdrbTvIKd7/t9l/0YlSMZ+tY0RR8PTOe9wlAsVXQe8FBakPU4Vp79BgpfzrLkd8MwBYJ07L+6WWrIDg6VXKEQqFQGEjQWrrHd3RhTY9Et5bZqJiaemNKN1aatA4d9O/QEZGRlI67mqceXcP42HNN0pEWdZKP75lgWHLE8u3DGHZdHlnRde2ExxMO0uPRNcwv1XSU3zzAmXRy/oZyAPYNd63zulbui2d78d6bWQAkzQt+r/zRMTYO9a9zY2y40J/06flkrXqAdjGVzOn1odu56IpLPGteTTlT900EIOlpSec9KnHECHw9VxdHZxL565Mh79dxxZFwVBsjWD/tT3Sz+O7Yy6sp565dU+ha7TvfIODjNWtrF1Kf/IpnrePpcKcWp+vwidXHnIO3GtKDDmCKj2Pl3PluPasOlp3+Kb3ftGIbcbVzWbfYQ17lxseeo//c+YYlR6RNyWHOhlvZ2O8DLx0/zp7Hfedm1Jt04mDpuSTO1LTlb0uvJ2mhMSFQeTXl/P3IlSSU5tF5rPYj53kuzjLkBf34Ufbf/rlFfbVj3KIdwyjXkF5EnbOx9FyS85oXZ0TReW+HkGdxicx+dIt195/7eq66PnOA1SkbDVbnjqlfX2oT62KJ3ROO3A3u9mrNjm240J+/H7mSzmNzsTbz+EH16abM/orHyn4FUO8vRolVCzgvqYimczAF+EFYLJAYrwWU+2B1ykZ4e2NA+zIyKN2cmECUxfePV8+IWDa95j9xo1rWkF+jbfvelFGIrbvoiHExp1P3TXQaOnA/F8f1H7dzCp3H5upy/I72H5fNC6OaZdSNvlcbIub9bawtHs3Ut7VkmR2/W8Kgqmm0WxVaoztsRY5XlEigz1W1rGFfTUew6Z9uZ05MwPbyBT7v+0695Y7VlnHBZuKuTx8DoM8j24LWOAh6R1ryn7VeyjHnn2DnbN+uhcxNjwCQPj2/2b8agWAb3I+X31pEL0t0w4UbwMig9K6fVrOg66fQhN79l0v68sXPegFgKt5DqLNHXc/F6OvfHFqS1pbKyyV9+eK6VKxFRbofK9BnaswLT9DlvQOkV+wFgvuGFHSj60ibjCj3/5jbqs0AWEvP+S0TTKRFBBQMHwhGBqX3iC4m1hS4wb1q+wQAOi6IwlxegzxtfDKCP1zPxejr3xxaktaWSrUtQlf3iMjsx7AVmhtzarutxJrqtwUDn51Gl7/t102TbiFj9QVxzx2mdbbMeT48As4DYebJgfxnYV0+Y4e9B3Rr+Zjj4zi+KonxccvwFcjvmhzx+FNvOTsGHaFQnTfleLVs858fQsJeiH/D2Pr2PJf0zZPosyJ8xgoArXNHzqxrZY2PW4Oj3h336usbhvvctnJ1F8Pr1JPhv97GFtuQkOtoCjNPDmR79o+JQZ8kiYujM+n6zAEX14dvg+v6THXWeaSzoBtdf0HpPT+cysPD/sXjCQcNDzhvlV9I2hvT2DKxLmb1p9/fRuFX7kMkWqOkWxmoC6Dfu7gfCS43tW4Gt08q+56JI+eaRbTzoePrf/QnqghnckTeY13AbnTvy9SWvTcryys6oevAk5yuSiJeJ90A3T8ykXFUCzBvmy+J6WNyO5f0zZPovsSC2LpDRxWB47hXvXvT6+rdca9O9OjMdJCR+rCudeoLz/v5pS47yEj9ieE6XPnb0utZ0/56n+t8PVcOvi1OJuZ9/bLSypIiGuy480w40tuNFHSj6y8ovffbNXyS2t8t4PjmpO95a/poZ6eHXtQePUav7HP8pM0MZJRWpd0/MpH8Qd1xLd27sf+R7l7bbvy/dADSDGpF1CQ5RhSri7KYfGQ4e5b2A7wD+V15poPmf/piVBrMc19XvCGJDgf0bWFGf7CdZBfbVDPiardzafuP1pg3hk9rzN+9Gu7UHj1G7wUXKb5L0FHzftBhyEnnj0j0B9sN11TfM2zu1NFNK+ibpNMUqtrDkTlDnd9T1pVg261PJ69KjlAoFAoDCWpLV147AFtGmduyMlsVd+TdjqW4Es/gp8cTDlI99d9sXqj/6F62Cxfo84j/15iLqR3Im7QEz2SC2NxWumtriK8/619vCzcQ9EiGqA9Lagr519VFi9xTkEXsiYZjt42kbb7m+c7aow3GEmWp4b20v7l1Xq65kAjA64d9+3Qd+zCc6mru2vUAa6/WRkz7T/919ByjjYiW5tsTElasytFalWkG35e+eDzhII97JHGldniQxJwhAW3fLq8SIOBopqAa3f33Wzg03H0Q86O1NkwTarB1t9AtttRtnZ7B8cFgxdrRJIfBTeELYbFgG9yPjhEbQi3FJ4U/7eKWjXj6qZ602hRemWDOjif7oGymxAQe+/RnZCdtcPofn95yGwBpD/jWHhmiV2Nr6Tk6jz3Hu99d42MUtfDB1KYN5YNSiBJ1QVcbKiJodcqYIWCjztmYW1Q3ot7Udt/69C17kn/ba3BbYMfo+Zn9x25rYOV1H/DGkUzws1VfeQ0g4RlAHyrCYfT6+rBGS0xttGmDbBcuICwWTGmpvPzWIrdQuHAL5G9pWIvPcmQwLA1zQ9aSqB3Y257EU5co9Zs3f0XK741pzMS8v43N79e9uRzeNooZnT73KhclbG7j+h6rLaO1MDU4lnVT0N3oOpIJelhaEa4DnIf76PVbJs7jJ21mAFpmjL9kDxXIr1DUz/GbIr3G9gUoH5TiluE55oUnOD+kslHTZAWKAS1dk1digiOIP+lpGRa58L5Gr89Y+jCpK44YPranLzqaW/POTQsB2PBdfzpGbPCq054fTiXjJS3WNJSB/Gema766px5dA8ChmjLumz6DmB25YXGtFcZR8YvB3JsdXpEh/uZri9lYycgJk53fu+zdT5cNbRj5+mSf5V3JOKHtM9CGTkjmSHMG8YfJaE++EjliD0tqj/ofAFsvIvYe48ezpwHuiQ+OmS4GRfp+7Y06acG6P98YkfVQpfU7OXVXSROttxdgbeLMqYqWS1WcyWswptR1D5L+QUnY/QDbLlxwm2jSClBYiGl/w9s29q3SUKObvnkSptxYuu8Kh/ZjeGItLCRhpZYN81zriRx+yHvgd6gbwd5cpf1idN1UZahOv4TxdDKXIq5JCS3huUrMMekW/9pSCKrRbb81gp5M8bu+z4paxNbwiwZol1fp7IF00MceBhJKOi78kjeiRrM446LXOlFlJj17D7YwakH6mhGgpZP1o30A7J0Unmm2eicWNYdwfa5CjUqOUCgUCgMJaks3YcVXJKwI5h6NQWzdFXCMndHUl9QQbn6xkt4WEoV7wkFJRXSzR9oPJSuTNwOQdU+iM55XERjh/FyFkpB0pCkuTZLmfek15kNn9B9AJNj8/ciVXssOH2lPWhiMEaBo+Sijq1B4kHBLnte08EbM46e4PBBShnpOAYVCobh8UB1pCoVCYSDK6CoUCoWBKKOrUCgUBqKMrkKhUBiIMroKhUJhIMroKhQKhYGEjdEVgruEIFcIyoXgoBD4nh8lxAhBghCss+s8LAR3h1pTfbSgel0tBCeF4LwQ5AlRzyAeIaQlXf8WpnW6EOQIQbUQrAq1nvoQggwh+LcQnBOCA0IEOseERtCTI4TAImXjhqEVgpHAC8CdwHagS/1bBIemaAUWAReBTsAA4BMh2C0l/xd0gS5cBvX6HPCAlFQLQV9goxDslJJvdZAIXPrXn5al9QQwFxgNRDdQNmg0VqsQWIC/A68CI4ERwEdCcLWUAc47JqUM6A9kAcinQO4FWQJyJcgokFkgj4H8LchTIN+0l78F5C6QpSC/BHllPfv+EuQDgWoJlVaQrUFeBJnmsuxNkM+Hm9aWVK8+jpMO8iTI8eGksyVd/5ak1eMYc0GuCtd7FWQ/kGUghcuyDSD/ELC2Rp7EHpDdQSaA3GqvoCyQtSBfABkJMhrkQJBnQA4GaQb5S/v2kfZ9LQa52P7ZbL85ngR5wF4hC0FGN7PC9dB6NchKj2PNAvlRGGptMfXqsv/FICtASpA7QMaGk84Wdv1bjFaPYwTT6OpRr/3xNrqfg1wXsLZGnsRDLt9vAnnQfhIXQUa5rFuCh+UHuQ/kCB/7TbI/ZDkgu4Bsb6+g7GZWuB5ah4M85bHsVyA3hqHWFlOvHmXMIIeBfBpkRDjpbGHXv8Vo9SgTTKOrR71GgMwH+YT98yj7/tYHqq2xHWlHXT4fBpLsnwulxHXqgh7ATCEodfwB3V3Ku+IY1fgVKTkpJUXAfOCmRmozQmsZ0NZjWVuguSOJX+716kRKrFKyBegGTAsznS3p+rckrXoRdK1SUgOMBW4GTgEzgXeBgOf2amxHWneXz8lozm8A6VHuKJAtJdkN7VBKSoTgmI99NJegawXyAIsQ9JESx+xJV0GzOyYu93r1hQXo1cRt4TK//rQsrXqhi1Yp+Q6tAw0AIfgS+EvAqhrZXP8eZDc0H8lmkH+0N9ePeZTNBHkUzUci0Jz6N4Ns42ffz4L8BmRHkO3s+w7YMW2w1ndAvm0vdy3IcyB/FKZaW0S92vXdBTIWzb0wGmQ5yJ+Hk84WeP1bklYLWkfXc2gdflEgLWGq9Uq7vhg0P/kh7P7fgLQ18iQcvYGlIP9iP6jXSdjL32B/4EvReqLXOk4C5KsgX3UpG4HmrC5F61F8GRefSxMrXC+tCSA/sBuFIyDvbqpOVa+aVpAdQG6ylztvf1h+FW46W+D1b0la54CUHn9zwlTri2gREWUgPwPZuzHaAh5PVwgKgClS8s+ANgghSqs+tBStLUUnKK16Ec5awyYjTaFQKC4HlNFVKBQKA1HT9SgUCoWBqJauQqFQGEi9cbojTePCqhn8uW2t8LdOaW06/rS2FJ2gtDaHS0FrS9EJqqWrUCgUhhL0oR09McfHcXxVEu1itKzUytVdiH/jK70Pq1AoFGGJrkbX3CeVfc/EkXPNItqZYwDISH2YeD0Pehlg7pNK7pMJftfH5rYiad6XBipSKBSBoqvRrUmK4+D1K4EYPQ/TbExXZVBwWzuf6zrs0sY3jv5gu5GS/GK6KoPcaW04dONSv2VeHNyLt6pG03GhcYa3dNIQzqf6dWO5EW51CmDp3o38B5IbLGeqhuQ/78JWUWGAqjqsWQMBOJ4V5bUuZV0Jtt25huppDpVjB1E4oM70RBVh6L0aapRPV6FQKAxEt5auJTWF/OvcZ92YeXIgbfPDqpMRgKKB8eROXexzXc/PtKm60j4wUpE3pqsyANg3tS2Hbn2t3rKPJxykeuq/2bzQu1WkBxW/GMyw32zjpS47Aio/MncMABcrM2m1PkdPaQFzMbWD33vAldyLFcxcMRYMbOnKawdwdJr2drBvuLfG1A4PkpgzxPk99kRN2NSrJxdHZxL565PkZnzkXDa3qK9h92o4oJvRLfxpF7ebeOm5JLZn/5j498OvE03U8zvQNrEcAFO/vtj2/GCQIm8Kxmruj/zbGjYMRiEsFmyD+3Fv9kdMjTvR8AZ2Prc/cPc8k0VRcT9kzh69JPrE1K8vtYl1DQJLcSW2ALeNEjbKB6XQ6nxXIvYew1pYqI9IF/bfb+HQ8GV+1+ff9hquUyPeUxCaeg0EObPIef0vV4JudE0xmv+2prW7f2/tQ6OJ2bQt2IdrFv60urJ70NsAZM0fS+QoQ2R5YWrTBmt0+L0hmBITWLBmMRmttHo8Yy2n2Oq7LlMjIogUEW7LVqdsZO6Kvmy+0thWTuX8Kjb2e8f5/YYfbubiAs/xvn3TMyKWTa9p/vRB/zONdqv0Nbrm+DhMkdZGbbM6ZSMvrurFF9elYi0q0gbsCgPMiQlEWWrclpXZqjhcmQiUG67H8fyLNrENF66uxlp6LijHDbrRPfKbAQCsn/YnIICTCSEtReu+56/g61vn2b+1DqmW+hi2Zha9Fxz0ue66fx3k8QTf60LNe2l/o3BhLeF4DxxflUTONYtobGf0o+1+oM+Xp1k6dIghrfFA6PppNQu6fgrU/cjekXc7pnGVhMLoOp7/9x6c10BJuGvXA3QeG6ZG19ZK+9/NEn43sCsF2UN45c7XgfDWmrcsk7/+bCEdzYEb23sKsih6tBtg7OuluVJgPX3G57p/3TeE3S93Z3XKRkM1OTAnJtD102p6RBczPm4Nrj9esaYo3rqQxJonbmHVwvn0jAiP++Hsx2ms7b+Mdh7Xfum5JNY+NBqAca+u9+naiRQRpEecAVNgESVG0CO6mFiT+1tNVW0EkcXHDdfi+vw73tTqY+3Vy5j48WQSbglslvX6CKrRPTFrKPePW++27FhtGWNeeIIue/fTuJck/ch/fghzbnuXUTE1PtdnLH2Ym8Z87dYxNKfXh0xedj9pU4zroMhblsnK61YwKDKi4cLUdVDV/qkTrXL012krPcfkp2dgjdAe7JQdJX59o/seiuLJTv9xWzbz5EC2Z/+YGAxwO1kszOj0sf0B8/4By4rZz7KHSrgj+3FMNfDT6Q13DA7/9Ta22Iboluzz8+TvSIvw1nqmpi2mTTsBeHP2GPbN/san1u4WE7a3I6iuTUG81D5sO9eMpqHn3xdpEa35efJ3bKb5rrCgGd0Ts4Zyx70b3V4hN1RE8NjyJ0heuYuDs6/iYucebtuEKoi/68CTTGxT7LX8jLWcYWtm0Wf5Eb4dmgwuN3JWtI2//mwJdy2aTvqTe7FdaO78fg3zwKAtZEU33MWTvnkSptzYuvjX9cbEv8rqauJWf+387kupqU0b9j1/BX/92UKvH49vi5OJeV9fgyuvHcD++y2YIq10NvsvlxbRmr/86C/MvH8s1tNn+HZCitv198VLXXaQkfoTQ5N9Xjzbi78tvZ6OaM9NzPvb2BI7hIzUn9BhyEkA/tN/HaC14P/R9xMAfpw0Df/pNPriuAf+0GYhEFgDQi8KsjWDW9/zv/COZY0yyI0laEY3cdQJnumw12u5LRIKnhjAK3e+7nUiLw7uxYrY0YYGnJ+ZPpS7k9b7XFdsFfRecJDa02co3jCU33e6wu2cBkVG8PWt8xl2YRZ9Fh2l9mjAE4A2ClNMDEd+M4BBMa8HVL7tP1qTsDK8gssdCSfWKMnXt85zc4/8vvAKAIo3JJFEga46StKiOXTjEvu3wP2ixRuS6Hl0ituytonlzo5VBx2GnKRy7CBA32QPR52992YWSR6JBPFvfEU8OHWM/PWYsIkQsHTvxv5Huof0HnDl3jFfeBncNRcSeXrLbYgqM+nZe9h+Uy9GxegXqaSSIxQKhcJAgtLSrfjFYAYlfuO1fFRMTb0B548nHOTxqYsNDTi/feq/ffai59WUc9euKXSt1jolkuZ9yXtkEXNftVMrQEdza/ImLWHkZ5Mx6dTSFW1iee/BeQE5+O8pyCL2hH6vQoFycXQmZUl1r47FmTaXmOK6Fs6LZ3vx3ptZAGE1PoSv6++JbcTV4N7Q5T/919FzjP4JNFuKtNno239/0W8ZR0v7bLshhHTiczuW1BTyJyWRN2kxnn70VTlDAUgz6B4QkZGUjruatKi6C7jmQiIAc9aNJ+1JzS8faLx2cwiK0R09ZxNPtw+sOb6hou7BdLgbHAHnMRsrDfGV+uLdc9fQeWyuW2df0rwvWdFa6yV+/EFjkhJMbdpQPiiFKFH/5a+WNbxc0peiR7sRVVVG7Yir/Za17Diga72KzH50feZAQJEJi7f8jIy/nwIwpGO1vsQXB3k15Uz8frLX9Q8VjqSTjhEbnMuyU98H4M5x00jz7R2r297HOQdSD8HGM0HKwYaKCFqdMta3a4qPY+Xc+W4Nmae3aBklDoNrFLoP7ejgUE0ZVdLEY8tnOZetn/YnullinQHnIydMdvbKBh0hMLdvT6Qp32tVKAO0Pakd2NsefO8/bKla1rC+Io4vxg3EdPYMtrcj+LzvO37L61qvwLAVOQH/6B66dSm9YicDkD6zI9TWYi0+q5u2QJi6b2JQQoGChWfSCcCd/54GUG/0TCDJPuHAb978FSm/D/1bjiPpxNypo9vySJP3m3AwbYRhRve+6TNovb2A5Au7nMvGnH+CnbONaUGa27dn6pdfcWNMCZ49qKEM0G4KL5f05YtxA3now0/oE1FEd4sJghDKYhQ5IxYBcGobzD89kiODQyzoEqGlJPuEC673oSupERHoaSMMM7qtztc4A+fPTNf8OU89usaow4NJkB5xhkjh7ScNVYB2U/ll3HewFkbHnPN5Pp6Me3U9b84eo3t4lidLz/lOOHCMrdzODNlJG5i9bRTHb4o0vMV71fYJACQ9LX368gqyh3DvmC+c3ztGNPBeH2LCJTGpvuc7Y+nDpK44Qq3BmmzFZ/nNxIe5Y9kGZzKJ633YEMG0EbobXV/JEVWa/5rxscFJq2sIU7++2F6+YG8R1uFIJhAvtQcDw1b8cXF0Jl2fOdBguY7m1vaOvcD8YlPjTrBv9jdsidUnkH/9nBH8Pe46AM7foLUEbum9h+3ZP6b1hl1MmD0La4TwmXDQ0dyaBV3/xR1rb8f0aPAHFZJ+3rTTN0+ix8va/WDbs8ttXd6yTHokF/G7Hr7jOT3302eF/iZk7jAt9nbO8+NJ9eGD9JWY5CBj6cOk/ueEYYauvuc79rDULdSyPmRtLWLrLs7UtAUCH5xJDxuhu9FtLUycH1JJlw1toLCQs/cP4Sc3fu9WxpmUkH9UlxujNjHa7vN0fwU/UNAJgLT6MnUMdI9FnSpn63dpoEOqrJ6B/DHvbyMGOHv/EIb2OATAwNjDfHzNYFLer0ug2GKtP4h/ZOJkw2IYh/Y4xMb707Uv92e6rfvr9UsCygJM3zyJ7kssiK2BDWnZHBzG//WBJ73W+UpMcqXbF5XU5hfoKc+Jr+fbQc8Pp9J3R6khEQL++NvS61nT/np+csP3rEze3GD5gGxEI9Hd6LYzx3Dw+pX0LJtK1KnOPk/WNSkhnKgcO8hpIByUWCvI3PQI6SfOBb2n27Y7l95vXw23BnnHBlA6aQj9pu5xu7aHxnzB5tl1P3QNBfEfmBBB39IMQ2ZBWJm8Gfw+dIG9QbT9R2vMG4P/5iArKrn5k8d45ybvLD4HjgQaWyTcf8d6nwZXz3vVH0XX1ng93w4dGS8VYd3v3ZFtJI4ZKvYcGULPa9Ody30lvuiFSo5QKBQKAzEuZOxW33N6eQalG0nvlNOA5kt1HQyk4heDqYozEX/PMa90ylNWSJ953O9oWuGK3rN2nL0C/quddxq4L1of0mKG9/2QBBl1y6+9Mo/jnXvTarceCoOLnkkptgsX6PPINjZ8159Bkd4+bnOHDpwc38cZcumLvJpyxu2cQvr0/KCNA9sQF0dnOp8pV8LxmUlY8RUJK+q++0p8efFsL2JzWwX92EExun8/ciXj4771OSJSfRgVlG4ur+HFs714tN0PbgNpe85gwK4fmjQTghFam8vmVwbTTqfRsABSn/yKpzvcxsQb62Y46BhxHnntTzBt24Os1bz15j6p5E5rA+A17dDpp3rSalNwR8IKZlLAGWs5S0uuAaDo0W6GjOTmSbfYUvbenmEPtfRvcEOR7OFrVogz1nLmnx4FtUbHKwSOuUMHCjO8Qy5XrB1Nsg4Zc0FxLyTckse4nVMosQaWxltiraDEWsG4nVMMCUqXOXv44me9yK/x3TJZnbKRn636ClNaKgvfWuTX4FbLGvbVdASbfi1GmbOHL65L5bOKdlTL4LSkDtWUYao1PiVpatwJFr61CFNaKuZOHTF36si+Z+I4dOtStzcfq7SRV1OO0EGjqVZyqKas2fsps1Ux+8QoNl8ZxeYro0I2Fc7qlI3s+N2SesuEItnD16wQAEtLruHI4PKQJ8DUR9HNvb3q9FhtGSb/GdfNImjuha73nSBz4SP2KdfrJ3PTIwDaq0+wBDSTR9v9wM8/+45elmi/ZV4u6Vs3BYqOWIuKWDp0CPu/yA/KbAv3TZ9B/IadhGLSll6WaP73s5VY7bFb2vCK7rHFB2sr+e8bJ2PK2xN0jfFrd3LfuRnOKXaaSktLoDEaX7NCtGTGvKANSatHpEXQjK619Bzpvz/HyNcnN1g2/cQ55zZG4Ss42pVIEUFahP/XecdsDLLQgBaOlFgLC5s124Lr7AIxO3KxVVcHWaQ3Gc+fpWfNVKDOh28WpnrdTkvPJfHelFGawdXhFVRWVxOzMZcRD05t9KwQPT+cSu+3tdabpbgSaxgn0LhqjW7ERJvBwtesEC2ZiHKp21CzQe1Is+7Px7Q/gHLBPGiAOIKj6xtp3xdGz8bgiszZw/HfZ/LjpIyGC3sQdc7mnAjUqAfQuj+fvksiAUi1PqjNUlsPzpkjtm7TtRVuu3CB6A27mTB7FjN+93ZASTmp6x6k79LzzvC1UMSWrp8zgpLZMfXeqxlLHyb2sNTiX0Oo1ReGzgzSDDwTaPROJjEseiFccB1pPxCMno3Bk1brc0I24n9TcDz46UszyCh8uN6ybfMl8e8bM8KTY5aL5+In8kz7hsunrysxJF64PgK5V1OXH6H26LGQG1pH0oErRl7fYKJ3MsllZ3ShLkhfoR+23bkkh2HoV8eFgfVGh9qIOWjoXg2XmIBA61WhkiMUCoXCUJTRVSgUCgO5LN0LCoVC4SDqnI25RX2d383lNbp27Cqjq1AoLmti3t/G5vddw930DQsVUoYiZF6hUCguT5RPV6FQKAxEGV2FQqEwEGV0FQqFwkCU0VUoFAoDUUZXoVAoDCTkRlcIIoVguRAcFoILQrBTCG4MtS5/CMF0IcgRgmohWBVqPYEgBH2EoEoIVodaiz+EoMzjzyoEr4Ralz9aSJ0mCME6ISi3P193h1pTfQjBXUKQa9d7UAiGh1qTL5pbr0GP0xUCi5SNDwiwIwAAE31JREFUSgm3AEeBEcAR4CbgXSHoL6W+86I3QSto8zfPBUYD/gffDTJN1OpgEfBNMPXUR1O0Slk3DYIQtAZOA2uDrc2VS71O0TReBDoBA4BPhGC3lPxf0AW60BStQjASeAG4E9gOdNFDm4/jGl6vAbd0haBACJ4Sgr1CUCIEK4UgSgiyhOCYEPxWCE4BK+3lbxGCXUJQKgRfCsGVvvYrJeVSMkdKCqTEJiUfA4eAawLVZpRWu973peQDoLip+ozSai9/F1AK/CvctbpwB3AGaHiO7BDobAl1av/huh34f1JSJiVbgA+Be8NNq53fA89Kydd2O3BcSpo8gHFY16uUMqA/kAUg94DsDjIB5FaQc0FmgawF+QLISJDRIAeCPANyMEgzyF/at4+072sxyMV+jtMJZBXIvoFqC4VW+/5WNVWjEVpBtgWZZ9/3HJCrw1Wrx3H+DXJOOOpsKXUK8mqQlR7HmgXyozDUagZ5EeSTIA+APAZyIcjoMNTa7Hpt7Ek85PL9JpAH7SdxEWSUy7olIP/gsf0+kCMaOEYEyH+CfC0IN7LeWoNpdHXRCvLPIH9r/xwsA6F3vSaDtILsGY46W0qdghwO8pTHsl+B3BiGWpNASpA5ILuAbG83ktlhqLXZ9drYjrSjLp8PA0n2z4VSUuWyrgcw095ULxWCUqC7S3kvhMAEvInmK5neSF2GatWBoGsVggHAfwELwl2rB5OALVJyKNx0trA6LQPaeixrC1wIQ62V9v+vSMlJKSkC5qP174Sb1mbXa2M70rq7fE5G61QCvAblOQpkS0l2IDsVAgEsR3NM3yQlwZgGVxetOqGH1iwgBTgitOlIYgGzEFwhJQPDTKsrk4Dnm6jNlcu9TvMAixD0kRLHJFpXQbM70YKuVUpKhOCYj300l/Cs10Y2178H2Q3NR7IZ5B/tzfVjHmUzQR5F85EIkK1B3gyyjZ99vwrya5CxzXlVM0irBWQUyOdAvmn/bAk3rSBjQHZ2+ZsH8j2QHcJNq8s2Q0GW11dG1Wmj7tV3QL5tL3ctyHMgfxSmWp8F+Q3IjiDb2ff9hzDV2qx6bexJPAVyL8hSkH+x34ReJ2Evf4O9EktBngS51nESaEb2VfvnHiAlWudZmcvfxGZWeNC12r/Pset1/ZsTjlo9tptDcPyPumkF+RrIN5ujUdWp272aAPIDtB+yIyDvDmOtEWgdVqUgT4F8GRe/a5hpbVa9Bjy0oxAUAFOk5J8BbRBClFZ9aClaW4pOUFr1Ipy1hjwjTaFQKC4nlNFVKBQKA1EzRygUCoWBqJauQqFQGIgyugqFQmEg9SZHjDSNCyvfw+e2tcLfOqW16fjT2lJ0gtLaHC4FrS1FJ6iWrkKh0BGR2Y/h31U5/yp+MTjUkkJO0MfTVSgUCgfW1hE83f4H5/e/x11HTAj1hAPK6CoUCkPo+eFU+u4oxRZqIQ1QkD2Ei5214V/ab40AIGHFV0HbvzK6CkULwJqljadzPCsKAFM1JP95F7aKilDK8omr1qrOtZRYK8jc9AgZLxVh3Z8fYnX+McXEcOQ3A3jlztcZFaMZ3Z5MAaD9txkU3NbOrXzKuhJsu3Mbf5zmS1UoFApFoKiWrgsiMpLScVdjjajreGyXpw31KbbuCommi6MzKUuKcH6PPVFDq/U5IdHSknG9th03naQ2v6DR+zB36EDhLb0B6PDxAayFhUFW6Rt57QCOTtOm8do3fDEAuRcrmLliLBU39KcqznfbyVwjiV+7E1ldbYhOB8dHaK3x3KkOrZA+8zjW02cM1REoltQUzozoQm2MYP20P9HNEutVpmhgvPN8HPz48DQSdjfheE0Veiliio9j5dz5ZLSqc/X3/Ex7vUjbGhpNcmYR3/T7wPn9noIsior7IXP2AFrvsLV1hNs2luJKbHt+QFGH67Ud9D/TaNcEo1tzRTe+yV4CwMDIaXT+G4YY3v33Wzg0fJnbsihho3xQCvdmf8TUuBM+tztUU8Z952YQszEX24Xmjl3eMKZ+falNjKaqS908j2es5cw/PQpqmzr/p/4U/rSL87pinx/13bI48qq60OqU9mwJHwFpvpYFgjK6LYzVKRuZu6Ivm6/UWhPDVuS49Q4DZO0ZS+QoY3WZ4+MgMrJxG1VXYy09p48gOyIyElN8HCTGY7Y/JTWtBaaYmGb5Q3f8bglXmx+m88pyXf2q5vg4TJFWr+U9I2LZ9NrSerd1lBnx4FSiN+zWtcVrTkzA9vIFPu/7jnNZma2K2SdGcWRwOVCu27GbiilGa1zVtPYOqZ33x7tpt+orUtA60M7d8xMO1ZTRM6KuFdzU+0gZXUVQOL4qiXcGLG/UNnfteoDOY/U1uqXjrmbl3PmYhaSXJRqAj377J0a3fYJuz33ZrH0Haz/1cXxVEjnXLIJmBFqtWjifCbNnEbf66+AJ86Drp9Us6PopEOVcdkfe7ZjGVRKOBhfgyG8GALB+2p9wtHD9Eb92J/edm+H2Q9fU66+Mbgvhqu0TAOi4IApzeQ2wJ7SCXDj7cRpr+y8jLaJ1o7Zbe/UyJn48mYRb8nRSBgnrD/Lo0Ue42DaCVQvn0zMilpErniB19RGa+8LbzRKLrVVQZPqlXUwl7czuBnfpuSTWPHGL83waomdELDaL3wSpoNAjuphYk2ZwHfdq0tMSa3GTZ1HXlYLsIbxy5+sATh/uoZoy7ps+g1bna2i/9wCu7xeyupqYjbmMeHCqs96bev2DZnTznx9C14En/a6PnhGl/IzN4HyxZtA6b8pxm+Bp/ZwRlMyO4aUuO0IjDPh58neNNrgAaRGt+Xnyd2x2aR0FG2thIaZNhbTu1JEqqXU4xR6W1B491ux9p657kPQPSnSLO81blsnKXivcls08OZDt2T+mzfZDzvMBSN88ibb/0K5BcaaN/Ntec9tO6mRzzfFxHF+VxPi4ZYB2fOe9uic8O3zznx/CnNvedYaFgebDnf/sQ8Rv0DoevR06YLtwgdbbC9zqvSkExegWZGsnMbFNsd8yV82dQKcXB4QsCqAlUpA9hN/1eLfeMjHvb2NL7BAmP9yalcmbDVLmzt+WXk/kQzU8nnAwJMdvCHOfVPY9E0dnc3D3m5hjalKcZkOY2rRh3/9v7+yDorrOMP7c/eBjWUCBBUEgsPLhWtNJE8TiiMF0jHEsKWaUfmAYSZUWddLRGlPHmWoczTSpJjMZqx0UtARKk1iTisZCOxXiEJVaNdW6EQ0gOhgRBAR2F3fZ7R+Xu7t39+73vbuLnN8/O3u5u3vuuYf3nHvO87znd3Pw0Q/2IzeUvUj6n/5UyI5fABLi2Z9RyxFzhHnMzQNW8F4sbkJD8Zdnqsyd7qKrK5DSELxKVEexql2XiOja857vjOlFZ+ZT0OUSEzviq9x6zMuqQIwLFcBgaR4eKS1XEtVhwrQa/twgnlDWnW92pASCVwvPOO3IGKbVnEPz8zlAgIJu/P4vURO2FAdUj52et3vhp25dD9/ok6LxzQtH4Mu8qD+hZOE4tfx9lormrQdzAAD9TUlIQhdMGi2Wn9oEUxg9Jku5Enh1wBJ1IXBQgfDP2gJdFIdw/U/VDcfiw4bF5kUzoQneLolAIBCeQHwa6VIhUkjmDyBNOghmPsdXHs4Bdq74yNwbFVwrAmp4+WqPOX/6aaRWC7cy/SSRtNd1PR1qykeJleZ4MiFRpqFjcbj5/equAsh7nD/d8cnRiwsAAFkT9WwcHkbmhgt++313GKxNRsxnno0WGWOCNd6aV5zBmGOywurNx+qGYwEAOz8thnK7l6NcL7S6PgXd8cEhzCgaQsnJMvwo9b8AgFwZPa9nO91QOZSEsCHXSw4ZtQN4d/aLKMmtd3kuYfJA5cxFspy9eNWuH8Xfur+LGAinXhArFHigsizUudsObXmwKJHlSLq/LR0hLfwvFIkiIzGam4YwylLGJo3ULNJ3Bz6F/I4QKxTQz0mGLkpqLqu7vyHOVEKfFA0A6Fgcbuf08ta84ghRZCQ0BSrU79lrVns0aaTYdawYANwOuLb35uORaIR5MVvGy0JazA/bzSvQdTvWAQDUv2BX5Ce/XApZi+ueWfueDl8FYDRESSQsAT2BX7hMHOU3SgSViwFA3/IMXPrtQfN7p+2QoiCOi4NxYAAieQTL7MEloOcbKjQUmgLVhBbUIgX71YfrkPZWcD1x9S3PQNvbbBeXK8SxMYBEghs7mDl2bvgwr1hjeDaDVad3DSPYVLUVaR7oa7nuzd63f4b4o57fF6LTncA4fy4++PMfzAJ6wtRDHBeH8i/P4cCalbj3xhjL7BEpMsLd4OItg6u+h/o9ewX/nUAx8/MxbE44OaEicbyoKbTppPCdrUg9csUjqR+f98broEvlzMXCavvHq1zZIc7zV/2xEb36KJffWxxdB77mhz3BJKFYWlNV5Xooq30X0E91xLExmPn5GMqnt8Kf97V3Iz0Huu31OtZxZ+0wVNSBZbIB9B5uQoHsplfaY18wSig7s4Mn7dDRNfuDZ3dVYMapW3b6Vus4UT69FfFidp2u7irA/W3pLPMKn6YTzSvz8eqeBtYx6ajJ7VG0dZ26Y0RxB6+Drm1GeFfQSTm4E3OwoW/KEnUhAIDaFwegy+Py+QpfAvopj0SCzQkn7f7ZhEZHr5GgWM62Gbtuh9KJc7jLe9cwgsJ3tiLx+k1OAb0vcBkYPGmHjq5ZVbkeyi96eB1A2JY1Vq3jTP7DjhN0nVobOeQ9eoS0XGSZV/hEFy1iJQTypC56tixA6ZpGAPzWadBNL2SfLYVILYdiQncY3hi8mj9/siaHftQ6WkX3vJROjOzfXOfMHrXuqbPYtacYad6uyApMWXc+AEBbm4hQoTpUgaZgh40iJB7zX1pHd3n4Wh6+v+wq59+Sz2h5VwPEXRpE+olydL5M5yLo3aRDgoE2P2mLcgEAdwqNiIq15F1IP1GOsG8lSGnWQdxsaZu25pXss6VIadH5VD6mDNNW0x1W7/goFtZtQWZVt8NOjDGlMNrnNTnNDg0/vtQpr0G3rDsfzf/L9uk7MqsNoFqDa9EgGNihuE6/LqNfe8dHsXB4C8Q6OroUfMfyD1cS2Y/OwjM4u104e60vMG0kS0DTC9NpL1EX4h8q9uPlEnUhbnUlOP18oIwc3jBYmoe55dfsHInMjg3ZPUO8j8ptsTY/PXiGDiudyw6gbUyP9NMVAMDaOcJ2d4nOFyrBzPNG/T2CFZS9gSmDeuLe949TyHj/Gxju99oZsBjGw0w4//Jeh09lbWN6/PhfE9fS89DrOiXmCAKBQPAjXo90Jf1a2rhghbY2UdDRy1Sjqm0h/pk6m3Xsudhu7Eu8hHhxBNpLDzr4ZHAgVihwb2XGxMo/ze8fzoJcLXBqLsBsRX2szUHBr9ntlNoXhywXu29wGTna9aP4yZW1mDnmztqEsFjvhLFo4wW7hEft+lGsurwW2Rs7BMlZrJsRgaeUvXjp6+U4lvVXyEVh6M8xAsiDUTViPq9p+Glk/Zyu63FYdkJ59BI97cDshMHAm+nERvkZKTLi3soMSDSzOOvLAnuUyxgoDt3Ox+3uOGSttVyLt3gddI3XvrZLlC3Y/FwACAa5LnODrWl7ZT5277RfeX0x8qpdcpRAo5+TjMvbD8BaZlP9yVKkuuFe44uQxotAo+3RLpefu90dh7ZMPatOPx56DjOK1II9qjszNUhSkgEAj5UK+jVKyhL7W9OuH0XJ1TJBy8rUqyghHncuGKEKAZ3ZzCbRTrz0EYzP55nfz9xxC7VpzXbfN2bS44OB2eh7PRkhF/k3nSRL5BNt0X2sDRRp288hi6f4FnQLaYGACg3F46jgCliOkB2/gLPH7edqq6oq0LnsMMcngoe7hhGInOfECRpUW27ip/vLzSL+EaMOt7Wx8HdCbn0EBXFCPG5uSAEAm6cbdsAdGKc741WX12JGEf/ZzzgxmnBDHw+Ae/+z/PBbyP/TLdYxNUcbuKGPx5nFSpgeBD5PdKd+BDqTCJuqtnhkoHAXEnTx5IvSgwVvROmBwnbHhkDtgtDw5rsYfkOEWDEzDHYsvctp2QAA9JSCH8oGAON9fahckAeIfJSLGE0Y7+vjp1A+smbjZkS0dSF1WJi2SoIuuEXpBN/wVZQeaGx3bNAZpAgNwC4IXDvT2qKqXI/kM1pk99Bzt0LvO8fCZAo6+RwAKKu7AQAqar1dbgcG691YrJFdUmNcwI08SdAFtyhdqEz7UwVbUTowuet056wTKDv8Guc8O18ovriHedsrYJBRaHiTeytwBsakIdGYoGzpgaGjy2+j28kAo8VV1kgw73YF5zkJ7VoAANV6mXVc6CcxEnQ5SD9RjtmXBifFY/BkYbLX6b+16Zh2RVjVhaGjCzEdXRDJZFgatRVGJ5sri8ZAT9VoNMSq7gSmToMJEnQBTG/XIv30WvN7axH3ZCGuVYp0WK4h5Fup3zLhu0NGvV6QrW2Eor8pCel3LPUpV4cgab9/VBdGjcatZC+TtQOb6hBzBIFAIPgRMtIFQLVeQZbV3m2TcW4spvocYqpdn0dwD3d2wiAQvIEEXYIghA0ZsbvP4qYTj+q92dmEQHjiIEGXIAj2Jo7Ai94JhGCAMpnI+INAIBD8BVlIIxAIBD9Cgi6BQCD4ERJ0CQQCwY+QoEsgEAh+hARdAoFA8CMk6BIIBIIf+T/lOfOfDrTFYgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 32 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "ds_test = create_dataset(test_data_path).create_dict_iterator()\n",
    "data = ds_test.get_next()\n",
    "images = data[\"image\"].asnumpy()\n",
    "labels = data[\"label\"].asnumpy()\n",
    "\n",
    "output = model.predict(Tensor(data['image']))\n",
    "prb = output.asnumpy()\n",
    "pred = np.argmax(output.asnumpy(), axis=1)\n",
    "err_num = []\n",
    "index = 1\n",
    "for i in range(len(labels)):\n",
    "    plt.subplot(4, 8, i+1)\n",
    "    color = 'blue' if pred[i] == labels[i] else 'red'\n",
    "    plt.title(\"pre:{}\".format(pred[i]), color=color)\n",
    "    plt.imshow(np.squeeze(images[i]))\n",
    "    plt.axis(\"off\")\n",
    "    if color == 'red':\n",
    "        index = 0\n",
    "        print(\"Row {}, column {} is incorrectly identified as {}, the correct value should be {}\".format(int(i/8)+1, i%8+1, pred[i], labels[i]), '\\n')\n",
    "if index:\n",
    "    print(\"All the figures in this group are predicted correctly!\")\n",
    "print(pred, \"<--Predicted figures\") \n",
    "print(labels, \"<--The right number\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "构建一个概率分析的饼图函数，本例展示了当前`batch`中的前两张图片的分析饼图。\n",
    "\n",
    "备注：`prb`为上一段代码中，存储这组数对应的数字概率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-10-20T07:22:20.152090Z",
     "start_time": "2020-10-20T07:22:19.983089Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure 1 probability of corresponding numbers [0-9]:\n",
      " [-2.4268928  -3.9816236   1.9360574  -5.2901382  14.079888   -5.757661\n",
      "  0.55549073 -0.10290891 -1.1255854   2.636149  ]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD4CAYAAABPLjVeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd5xU1fnH8c+zu3TpzUaRpqBIR1GaYBRFxcQuEaxogvVnSWLibxxb1Pys0RgTY1ckGhMLRkWjFAGlLiBFel9Y6u5Stj6/P+7duK4Lu1PPlOf9es1rl5lbnh3gu2fOPfccUVWMMcbER4brAowxJp1Y6BpjTBxZ6BpjTBxZ6BpjTBxZ6BpjTBxZ6BpjTBxZ6JqEIiJfisi1MTp2WxEpEJFM/8+tRWSqiOSLyGMicreIvBCD844WkU+jfVyTnLJcF2CiS0TWAteq6meua0k0qroeOKzCU+OA7UAjjdKAdRFpD6wBaqlqiX/eN4A3onF8k/yspWvSWTtgSbQC15iasNBNYSJypYh8JSJPiMhuEVktIqf4z28QkW0iMrbC9iNFZL6I5Pmv31vpeGNEZJ2I7BCRe0RkrYic7r+WISK/FpFV/ut/F5Fmh6htlIgs8M+1SkRGVLFNRxH5j3+87SLyhog0qfD6r0Rkk989sFxEhvvP9xeROf6xt4rI4/7z7UVERSRLRF4GxgJ3+V0Op4vIvSLyeoXjDxSRGf57t0FErqzB+zTV/7rbP+4A//2eXuG4p4jIbBHZ4389pcJrX4rI/f7fW76IfCoiLQ7+t2ySjYVu6jsJWAg0B94E3gL6AZ2AnwPPiEj5R+69wBigCTAS+IWInA8gIt2APwGjgSOAxsBRFc5zM3A+MAQ4EtgFPFtVQSLSH3gVuNM/12BgbVWbAr/3j9cVaAPc6x/jWOBGoJ+qNgTOrHCMp4CnVLUR0BH4e+UDq+qVeB/5H1XVwyp3x4hIW+DfwB+BlkBPYEF175P/swA08Y87s9JxmwGTgKfx/k4eByaJSPMKm10OXAW0AmoDd1Tx3pgkZaGb+tao6kuqWgpMxAuu+1S1UFU/BYrwAhhV/VJVF6lqmaouBCbghSjAhcAHqjpdVYuA/wUqfiy/Hvitqm5U1UK8cLxQRKq6bnAN8KKqTvbPtUlVl1XeSFVX+tsUqmouXkCV11MK1AG6iUgtVV2rqqv814qBTiLSQlULVHVWGO/baOAzVZ2gqsWqukNVF9TgfarOSGCFqr6mqiWqOgFYBpxbYZuXVPU7Vd2P9wujZxj1mwRloZv6tlb4fj+AqlZ+7jAAETlJRL4QkVwR2QPcAJR/tD0S2FC+k6ruA3ZUOE474J/+R/HdwFK8YGxdRU1tgFVVPP8DItJKRN7yuxDygNfL61HVlcCteOG+zd/uSH/Xa4AuwDL/4/s51Z0rlBqreZ+qcySwrtJz6/jhp4acCt/v44cX/0ySs9A1Fb0JvA+0UdXGwJ/xPuIDbAGOLt9QROrhfTwutwE4S1WbVHjUVdVNVZxnA97H/ur8Hq81faLfVfDzCvWgqm+q6kC8wFfgEf/5Fap6Gd7H80eAd0SkQQ3OV9MaD/U+VXdRbrNfb0VtgareJ5OCLHRNRQ2Bnap6wO93vbzCa+8A5/oXgWoDQSoEIF7wPCgi7QBEpKWIjDrIef4GXCUiw/0LcEeJyHEHqacA76LUUXh9wPjHP1ZEholIHeAAXou91H/t5yLSUlXLgN3+LqUhvRNef+/pInKxf+GtuYiUf8w/1PuUC5QBHQ5y3I+ALiJyuX/cS4BuwIch1meSlIWuqeiXwH0iko/XZ/vfC1Cq+i1wE96FuC1APrANKPQ3eQqv9fepv/8svIt4P6Kq3+BdKHoC2ANM4cetP/CCvbe/zSTg3Qqv1QEexhtnm4PXqr3bf20E8K2IFPh1XaqqB2r6Jvg1rgfOBm4HduJdROvhv3yo92kf8CDwld/VcnKl4+4AzvGPuwO4CzhHVbeHUp9JXmJDFE04/BEPu4HOqrrGdT3GJAtr6ZoaE5FzRaS+3z/6f8Aiqh7qZYw5CAtdE4pReBeCNgOd8T6220clY0Jg3QvGGBNH1tI1xpg4stA1xpg4stA1xpg4stA1hyQibfxbXpeKyLcicovrmoxJZnYhzRySiBwBHKGq80SkITAXOF9VlzguzZikZC1dc0iqukVV5/nf5+NNZHPUofcyxhyMha6pMfGWoukFfO22EmOSl62RZmrEv+33H8Ctqprnup5k5/eNX4c3adBfVfVJxyUlnLlz57bKysp6ATiBxGwglgGLS0pKru3Tp8+2mu5koWuqJSK18AL3DVV9t7rtzaGJyAl4gdsfbxL5j0VkkqqucFtZYsnKynrh8MMP79qyZctdGRkZCXfxqaysTHJzc7vl5OS8AJxX0/0S8beHSSAiInhTMS5V1cdd15MiugKzVHWfv2LwFOCnjmtKRCe0bNkyLxEDFyAjI0Nbtmy5B68lXvP9YlSPSR2nAlcAw8RbSHKBiJztuqgktxgY7M/RWx9vCsk2jmtKRBmJGrjl/PpCylHrXjCHpKrT+eFk5bHldWW0wFsMskWl7xsBtYBMILMBBfv30aAW3ppoJf7X/XgT8mwCNgIbVf87kXlCUNWlIvIIMBlvkvZsvPpNAlm5cmWt0aNHH5Obm1srIyODsWPH5t5zzz017rs9GAtdE38imcAxeB+zKz66AE1repgD1N1ODdYmE2EvFUK4wmMRMFfVWzsunlT1b3jdNojIQ3495lBE+kT1eKpzD/VyrVq1eOyxxzYOHDhw365duzJ69erV7eyzz87r06dPSBPiV2aha2LPWy59MDAIb2XbLngrP8RLA/+cXap4rViEhXgrXcwEZqlWv2hmpESklapu85d6/xkwINbnNKFp165dcbt27YoBmjZtWtaxY8f969evr22haxKLSAbQHS9ky4O2qhWBE0UtoI//GA8gQi7eWORZ/mOmKvuifN5/iEhzvC6R8aq6K8rHN1G0fPny2kuWLKk/ZMiQgkiPZaFrIueN4T0LOB9vfbJmbguKWEu8dczKl27fJ8JHeGuhTYpGAKvqoEiPYeJjz549GT/72c86PvzwwxuaNWtWFunxLHRNeLx5GEYBFwM/Aeq6LSim6gMX+o+9lQI47v3BJn4KCwtl5MiRHS+66KKdY8eOjcoFWQtdU3Ne18HZwJX+13pO63GjAXCR/9grwiS8AP7IAji1lJWVcemll7br0qXLgXvvvXdrtI5r43RN9URaIPIrYBXwAXAB6Rm4lTXAa+m/A2wT4UURTnRck4mSyZMnH/avf/2r+fTp0xsed9xx3Y477rhuEydObBzpca2law5OpD/exaWLSe3ug2g4DLgKuEqEL4AngQ9VibgP0PiqGeIVbWeeeWaBxuCc1tI1PySSgciliMzGu4I/BgvcUJ0GvAcsF+F6kbgOjzMJzkLXfE/kZ3h3R00A+jquJhV0Av4MrBbhNhHquy7IuGeha0BkJCJz8WYSC2nyDlMjRwKPA+tEuFvE+sPTmYVuOhMZjsgM4EOgt+ty0kAL4EFgqYjNKpauLHTTkUh7RD4EPsNuP3WhHfCuCB+L0Nl1MSa+LHTTiUgtRH4DLAFGui7HcCawWIQHrb83fVjopguRQcAC4CFsjG0iqQ3cjdflcIHrYsyPlZSU0LVr126nnXZap2gcz8bppjpvUpU/4N1FFr95cU2o2gLviPApcJMq37kuKNFIMLpTO2qgZmNwH3jggdadOnXaX1BQkBmN81pLN5WJnIe3ZPpVWOAmizOARSLc5LoQA6tWrar1ySefNL7uuuu2R+uYFrqpSKQeIs/hDdBv6bocE7LawNMivC1CI9fFpLPx48e3efTRRzdmZEQvKi10U4wEpftHnXgbuMF1LSZiFwJzRejlupB0NGHChMYtWrQoGTRoUFTnUrbQTSESlGuBr0ddRo9dddnjuh4TFZ2AmSL2SzTepk+fftjkyZObHHXUUd2vvPLKDrNmzWo4atSoYyI9roVuCpCg1JGgvAL8FahXksnR/caxxHVdJmrqAM+J8KYIh7kuJl08++yzm7Zu3bpw06ZNi15++eXVJ598cv577723JtLjWugmOQlKS+A/eBPT/NeqZgz41elMc1OViZHLgDkidHddiAmfqCb0svLmECQoXfFu4e1Q5QbKgW/+yoZ+m1PzrqdMSraXkVntasApaD9wlSoTXRcSS9nZ2Wt79OgRtVEDsZKdnd2iR48e7Wu6vbV0k5QEZTgwg4MFLoBQd/BVZOytFfVFFY1b9YA3RRjnuhATOgvdJORfMPsYaFLdtgdq0XHIVcyLfVUmzjKA50W403UhJjQWuklGgvJbvAtmNb6bcO6RDHx8ADNiV5Vx6FERHnRdhKk5C90kIkG5F3ggnH1vP4MTljdnfXQrMgnibhGeEbG7DpOBhW6SkKDcDwTCPwCN+o2joCiD4uhVZRLIeOAVEaIyP4CJHQvdJCBB+T3wu0iPk1+HbueMtm6GFHYF3qQ5tiZbArPQTXASlD8Av47W8SZ3YPDrJzInWsczCed84ENbEig6tm/fnjlixIgOxxxzzPEdOnQ4/rPPPmsQ6TFtascE5vfh3hHdgyJjfkr7oWvYenQ+raN6bJMoTscbUnZBSi0BP2VOVKd2ZEjfaqd2HDduXJszzjgj7+OPP1594MABKSgoiLihai3dBCVBuZpI+nAPQYUWvW9gc6mk0H9IU9n5wNPROJCINBGRd0RkmYgsFZG0WOJp586dGV9//XXDW2+9dTtA3bp1tUWLFqWRHtdCNwFJUM4Eno/lOXIb0GvMT+024RQ3XiQqXVNPAR+r6nFAD7w5mlPesmXL6jRr1qzkoosuat+1a9dul1xySbu8vDxr6aYaCUpP4G3i0PXzZncGftSJhbE+j3HqoTFnbg97GSARaQQMBv4GoKpFqro7WsUlspKSElm6dGn98ePH5y5dunRJ/fr1y+65557DIz2uhW4CkaC0BT4CGsbnhGSOuozmO+qRFv+J0o+W3nzB1qmv3r32DabMCbdLoAOQC7wkIvNF5AURifhiUjJo3759UevWrYuGDRu2F+CSSy7ZlZ2dHfECoha6CUKC0gCYBBwRz/OWZHJUv3Esi+c5TTxowcu/XjvvqZs2DMGbGvKfTJnTNowDZQG9gedUtRewlyiOpklkbdu2LTn88MOLsrOz6wB8+umnjY499tgDkR7XQjdx/Bk4wcWJ1zTl5NvPYKqLc5voy8zQLTOeXbZp7Igd/So83Rp4nylzQm2lbgQ2qurX/p/fwQvhtPDHP/5x/ejRozt06dKl28KFC+s98MADWyI9pg0ZSwASlOuAn7us4fEBnHTRtyw/eRPH1mT7q/HmlGwFLPaf2wlcAqwF2gN/B5pWse8IYBYw0D9GudHAIuAcvHXiAe4HTgRG1fxHSWsN6pYuW/LKt43bti6q6u+xB94F2hr/W1PVHBHZICLHqupyYDg4miC/BkO8ou2UU07Zv3jx4qheOLSWrmP+hbOoDO2JrBDqDL2KWgW12FuTza/Em+asoofx/keu8L8+fJB97wReq/TcwgpfpwF7gC3AN1jg1tTRLYu+yflndpu2rYsO1UU1milzrgzx0DcBb4jIQqAn3/9ONGGw0HVIgtIIb6RCXde1ABRm0WHg1cyvybaDgWaVnnsPGOt/Pxb410H2Hc6PrxTWwpuZuwwoAjKB/wXuq0kxhiE98qeunbiw72H1ymrSffAMU+bU6BMNgKouUNW+qnqiqp6vqrsiKDXtWei69SLewoMJI/sIBj56Kl+Fs+9Wvr8KeASwLYR9uwJt8ToLLwZWAgpRWgb3aryOkIpd5vfgdVz0BM4ANlex3xf+6+WPunz/q2S0v//dFba/H+9XTzxp6c0XbJ3y5VPLB2dm1Pj/cwNgIlPm1I5lZaZqFrqOSFDGAmGPn4ylX53OiUtbsC7e530SWADcjheJ9wEP4oXwXyM68pX8uDPkTrzOjAV4vchVtalP819fgLcMXX28gE6UzpAfjFAIVQ+83xImzix0HZCgHA484bqOgxIa9r+OfYWZFIWyW2u86MH/2irM078H9MUbm7QY74LcaxDBmkNVdYY0qvD9Xqh2Ktp3gLPwgtd9Z8hBRiiE6g6mzDk1akWZGrHQdeNZqr6wnzAK6tD1rNHMDGWf84BX/O9fIbw2XzHePad34oVseRSWx1t0/RZoA7xB9YH5Ft5ivBDrzpDqNKhbumzNWwtlwPF7a9wvexAZwKtMmWPLuseRhW6cSVAuBH7muo6a+KIDQ17qyTdVvXYZMABYDhyNd4/or4HJQGf/a/kI+jnAtRX2HQRcBHzu7/tJhdeexbsIVx+vx1SB7sCp1GBBuJA9CGzA65995hDbbcEbzHZmhedi1xlyKOUjFNq0Ko74dlRfB+D3UTpWyrn//vtbde7c+fhOnTodf99994X74e0HbAn2OJKgNMMb45g0UyqKsmPtExS1zYvvnXI1EdoS7Gvx+m4XV/HaOmDkQV4Dr+39LfCXKl57Dy98L8frYpiA153xMd6vjugZ0iN/6udPLB8YwgWzmioF+jGkb41GrsRL5SXYRYjq1I6qHHLc7+zZs+tefvnlHefNm7e0bt26ZUOGDOny/PPPr+vevXthpTptCfYE9gRJFLgAKjTvfQNbU28ayBUVvn8fOO4Q207g+66FiuLVGaKlt14Y8giFUGQCf2LKHFtjrYJFixbV6927d0HDhg3LatWqxamnnpo/ceLEiD9wWejGiQTlZGCM6zrCsaM+PS+/IJmngTxYZ8gJeJ0Yn+KFJ/y4M2QtXhdEVQME4tEZogWv/GbNvCduDGuEQihOxhtbZ3w9e/bc//XXXzfMycnJzM/Pz5g8eXLjDRs2RDzMzm4Djp/HXBcQib8fz8ArFpJ9znf0cF1L6CZU8dw1B9m2L/BChT+3BzYdZNtbK3wvBzlP+DIzdMv0Z5blndxtbyQjFGpkb+n+5Zct+e2YD76c9o4GdE+sz5cMevfufeCWW27JGTZsWJf69euXdevWbV9WVuSRaS3dOJCgXACc4rqOiAiZ519Kq9z67HRdSjooH6FwcreIRygcUpmW7Xh4/ctTG00b0vmDHdMG4w3pML7bbrtt+5IlS5bOmTNnebNmzUo7d+5ss4wlOglKLQ4+DUFSKc3giD7Xs9J1HamuTavCaI9Q+BFVLfli15wpzb86Pes3q58dXIaWZ8HNEpRjYnXeZLNp06YsgBUrVtSeNGlSk2uuuSbiRod1L8TeL0mwW30jsaEx/W86i6l//DeDXdeSiob2zJvy2ePfDYrRBTMAcop2zD1r4c1NFhR8V1U/cR28O9WcznqXKM4777yOu3fvzsrKytInn3xyfcuWLSNeI82GjMWQBKUxsApo7rqWqFKKpr3IqoEb6OqyjNCGjCU6Lb31wm3TY3nBrKiseP34FY9seWHLeydVs2kpcJwG1OmnmspDxhKVDRlLLLeQaoELINQePpZ6+bUpcF1KaojtCAVVLXglZ9KXjaYNaV2DwAVvCJn17caIhW6MSFDqATe6riNWirJoP+Basl3XkewyM3TLzD8t2zTmzJ1RH6Ggqrp476qv2s46p+DKZfcOLdTiOiHs/nPr240NC93YuRpo6bqIWPq2Fac+MCi8aSANNKhbujRWIxTyS/YuOWPhjYu7z7701I2F28K5IJdFmqyFFm8WujEgQcnEuyk/5d0zjJ6LW7LGdR3Jxh+h0DbaIxRKtSw3sOb56Y2nD+362a5vukd4uCslKG2iUlh4ysrKyhL6Ljm/vpDu1rTQjY2LgfT4aCY0OOk6ig5kUlj9xga8EQpr3lpU01UeakRViz/ZOXNKs+nD6ty37oWBWv1clTVRG7gjCscJ1+Lc3NzGiRq8ZWVlkpub25iDT9pRJRu9EAMSlPl4Sw2kjUHrmDr1pfgOI0u+0QtaettFW796fPzGqL5PGwu3zR6RfVOLb/etjsUv+j3AkRrQ8KczDtPcuXNbZWVlvYB3v3YiNhDLgMUlJSXX9unTp8YLpVjoRpkEZRjerIVp5y/v8/V186jJ1fGoSLzQ3YA3vUYOXkaMwxvAAqD5r/xmzbJzT1nb7+pH7mfV5o3UrV2bF++6hxM6dCJ39y5++rs72V2QzwPX/ILzBw0FYNRvb+e5237NkS1+fHmgsKxozXXLH9z+2taPYn2b8DUa0BdjfI60kYi/PZLdONcFuHL9uXRZ06TKxcbSRBbeFBtL8RaZfxZYUj5CYfOYM3f2e+j1l+jZqQsLX5zAq78Jcssz3pQcEz7/hLEjRjLzTy/yh4neWskfzJhK787H/ShwVTXv+c3vTmk4bcjRcQhcgOvjcI60YaEbRRKU5sD5rutwRYWmfa5ne4kQ8V07yekIvNUkwFvvuCt1ai1fX3GEwpJ1axje28vJ49q1Z23OFrbu3EGtrCz2FxZSWFRMhmRQUlLCk+9M4M5Lr/jv0VVV5+Uvm3bUzLMKb/ju90OKtaRWnH6w/hKUtOouiyUL3ei6Au82yrS1qx4nXnwx013X4d5aMjK+Llo9oXnziiMUenTszLvTvgDgm6Xfsi4nh42527h8+Ag+mT2LEXfdzL1XXsef3nuHMWeMpH7dugDsLslfNHTB9Uv7zL1i0JaiHS6GIlprN0qsTzeK0vECWpWUsncnkv3TZbFdNCzx+nTLFVC/Tu+Cl39zbf2Lhg77QcMmb28Bt/zxMeavWE73Dp1Ytn4tL9z5O3p06vLfbXbl53FJ8G7eve9RfvHkI/unb5y/b+2ALc1oE5URCeHKx7ugZnchRshCN0okKN35fm3utJdZRs7m/6NWq32xuw06MUO3sPSI5qfm3XHJoKb/c/HoQ26pqhxz6SgWvvgmjRp8vzbkbc88zrmnDC56de6Haybs+bR90QnFdXgLbyV5t67QgL7uuohkZ90L0XNF9Zukj9IMDu99A6vLvOUU0kRpfu8u5+6+eGjngwbu7vx8ioqLAXhh0r8Y3KPXDwJ3xcb1zF/73Y5fFj+85ZUdk44torgOApTEo/5qJcWCqonOWrpRIkFZg7fMgKnghtlMeW5SlWvdRCyRWrqZGbrlyRtfKrnp6WvadO/QiQzxegIeum4867fmAHDDqAuY+e1Cxjx0L5kZGXRrfwx/u+semjZsBMD+0sJVHW4a1Sxn+I6mNAcK8FZ+LwROA7q5+Ml+YB/Q0sWY3VRioRsFEpQT8NboNpUpRV++zMoh66IfGYkSug3qli5d+uripuHe0lumuueZTRMX3L7qyVNLtDTR57i+UAP6D9dFJDPrXoiOc1wXkLCE2j8ZQ4O82uS7LiUW2kYwh4Kqls3KWzz18Blnltyy8rEhSRC4YF0MEbPQjQ4L3UMozqTdSdel3ieB03rlTVkd5hwKO4r3ZJ8y/5oVA+ZdNTi3eFcyzbk8UoIS8Yq46cxCN0L+DREDXNeR6Ja15JR7h6bK+F0tvf3inKn/eeK7IaEuq1OiJZtvW/n4jBZfnd5jVt6imC46GSONgWGui0hmFrqROxt7H2skOITe2a1Z7bqOyGj+a3evmf9/vwxt0hpV3f9O7udfNp52WpMnN05I7pWh4SeuC0hmydCHlOhGui4gaQj1B1xLyY5HOFCvhLquywlVZoZumfHssvz+Xff2DWW/FfvWzxyx8OY2qw9sGhqj0uLNFiWNgLXQIhezhQRT0f5adBk+lm9c1xGq8lUe+nfd26X6rT37Sg98N2rR7Qu6fHPBgNUHNh0dy/rirJcE5bDqNzNVsdCNgASlPRDVmf/Twcw2DH6uL7Nc11FToY5QKNOynX9Y/9rURtOGdHx/x9RUvC08EzjVdRHJykI3Mie7LiBZjR9J11VN2ei6juqEMkJBVUun7Z4/peVXP8m4a/XTg0spy4xHjY7YJ7wwWehGxkI3TCo07juOXcUZCXKD64+ENkJhW9Gu+f3mjlk9eMG4ITtL8prEo0LHrF83TBa6kbGhYhHYXY/uF1ySiKsJ13yEQnFZycZffPfwrNYzzug1t2BZ53hUlyD6SVDSehrTcFnohsn/B5eK/XVx9UEXBr3djXmu6yiXmaGbv35u2Zafn7HzkCMUVHXv61v//WWj6UNa/HnzP9LxE09t4DjXRSQjC93w9cL7h2ciIWRceiFH5zQg13UpDeqWLl07cWFmdSMUluxd81W7WefmXbH0f4ceKCtKuqFvUXS86wKSkYVu+NzP+ZQiyjJo1esG1rmcBrJ8hMLRLYtbH2ybgtJ9S0dk37zw+NkXn7qhcOsR8awvQVnohsFCN3w1Hq9pqpfTkL7XncdUF+ce1vvQIxTKtGz7A2v/Nq3xtKHHfrJr5onxri+BneC6gGRkoRu+dLpoEhcv9uKUz4/h2/idUUvvuCRn6uePVz1CQVWLP9/1zZRm04fXumftnweVofb/5YespRsGm083TBKURdhv+qjLKmVD7qM0alJI4+q2jWw+Xc1/7e41yw92wWxzYe6csxbe3Gzh3pUdwjt+WigDDtOA7nddSDKx39xhkKAI0NF1HamoJJM2/cexJJbnONQIhaKy4nVXLQt+c9TMs/ta4FYrA0jGmdKcstANTxugnusiUtWK5gy4ezjTYnHsw+pVPUJBVfP/tuW9LxtOG3zEyzkf9o/FuVPUUa4LSDYWuuGxVm6M/X4gfecdwcpoHrNd68Kvc/6Z3a7iCAVV1eyCFdOPnnn2/muXPzC0SEtsGGBobBRHiCx0w3PQYUUmSoR6p14N+7KISn/h8N55U1ZNWNSvQd2y+uXP7Skp+HZ49i+X9Jxz+cDNRdtbReM8acgmfAqRhW54kml5laR1oBadhl7FnMiO4o1Q+KzCCIVSLd3629V/mt50+mndvtg9x67AR8ZauiGy0A2PhW6czD6KQU+dxMzw9tb813+3Zv4ffuHNoaCqRZN2TP+y6fRh9R9a/9JABYlmrWnKWrohspUjwmOhG0e3jqDbWSvY0GUnbWq6T2aGbp7x7LKC8lUe1h/I+ebMhTe1XrZv7dCYFZqeLHRDZC3d8FjoxpPQuN848ooyKK7J5hVHKBwoK1p92ZLfzmk369z+y/atbRfrUtOQXd8IkYVueCx04yyvLsePuqz6aSDLRygc1aKo7nmOlmMAAAsDSURBVLOb3p7ScNrgtm9t+zSkNc1MSNJ5wp+wWOiGp6nrAtLRx50Y8tbxzD3Y68P75E1Z+ebCPkuKFs09fMaI4htXPDqkREutCy22bIhdiOwfZHhquS4gLQky+gLaDlrPtqPyqTDES0vvujTnq19fs7Tp0Oz/WfFVXvYgd0WmHQvdEFlL1ySVsgxa9r6ejd9PA6n5z9yxdFrp6XdmNftq+Ilf5WV3dVth2rHQDZG1dMNjQ40c2nYYva86nyl1Pixuecntz677VYN7Bu/duL/ahSNNTFjohshmGQuDBGUe3soRxhWlpPaB1luK6m2t8TAyEzOZGtAy10UkC+teCI+1dF0TsixwE4Zd4wiBhW54LHSN8ShQ5LqIZGKhGx77KGWM54AGrI8yFBa64cl3XYAxCcJWjQiRhW54drouwJgEUeC6gGRjoRueXa4LMCZB7HFdQLKx0A2PtXSN8ex2XUCysdANj4WuMR4L3RBZ6IbHQtcYz0bXBSQbC93wbHddgDEJYp3rApKNhW547B+aMR77vxAiC93wrHBdgDEJYq3rApKNhW4YNKC7sS4GY8BauiGz0A3fStcFGONYIZDjuohkY6EbPutiMOluvc27EDoL3fBZS9eku4WuC0hGFrrh+851AcY4Ns91AcnIQjd8810XYIxjFrphsNAN33dAnusijHHIQjcMFrph8i8gzHVdhzGObNKAbnNdRDKy0I3M164LMMYRa+WGyUI3MjNcF2CMI3NcF5CsLHQjMwNvYT5j0s1/XBeQrCx0I6AB3QEsdV2HMXGWB8xyXUSystCN3MeuCzAmzr7UgJa4LiJZWehGbpLrAoyJs09dF5DMLHQjNw0br2vSy2TXBSQzC90IaUCLsd/8Jn2s04DaLfARsNCNDutiMOni364LSHYWutHxETZ0zKSHCa4LSHYWulHg3w5pN0qYVLcR7xqGiYCFbvS86roAY2LsLZu0PHIWutEzETjgughjYsi6FqLAQjdKNKB7gH+5rsOYGFmmAbVJbqLAQje6XnZdgDExYq3cKLHQja7JwCbXRRgTZaVYgyJqLHSjSANahl1QM6nnnxrQ9a6LSBUWutH3PGCTgZhU8pTrAlKJhW6UaUDXAW+7rsOYKJmnAZ3uuohUYqEbG39wXYAxUfK06wJSjYVuDGhA5wOfua7DmAhtA95yXUSqsdCNHWvtmmT3rAa00HURqcZCN0Y0oJ8C2a7rMCZMO4AnXReRiix0Y+sh1wUYE6ZHNKA2OX8MWOjG1tvAbNdFGBOiLcAzrotIVRa6MeTPyHSn6zqMCdEDGtD9rotIVRa6MaYBnQJ84LoOY2poLfBX10WkMgvd+PgV3v3rxiS6e/11/0yMWOjGgQZ0KfA313UYU40Z2NwhMWehGz8BbKl2k7hKgBtsZYjYs9CNEw1oDnCX6zqMOYgnNaCLXBeRDix04+svwFTXRRhTyXrgXtdFpAsL3TjyP7pdi62lZhLLLRrQva6LSBei1oUTdxKUXwEPu64jJrbzw4ktdwGnAfuBZYAADYDzgUaV9t2Nt7xnmf/oD/TD622cgNcj3s9/HuB9/89HxODnSB/va0BHuS4inWS5LiBNPQZcDPR2XUjUtQB+4X9fhveTdgXqAsP852cBU4BzK+17GHAN3r/KQuBPwLHAZuBIYDTeFPH9gRxAscCNTC4wznUR6ca6FxzQgJYAV+NFS+paDTQDmuCFbrlivBZvZVl83wwoxQtVgEx/n7IK2/4HrwVtInGdBnSr6yLSjXUvOCRB+QVeey41/QuvJXqS/+fP8eZdqwNcidfNUNke4A1gJ3AGXqu2FPgnXtfFKUBtvJbu0JhVng7+qgG1Vq4DFrqOSVAmAJe6riPqSvC6FsbjdRtUNM1//VAt1Ty86bMvr7R/KfAacBnwBV5I9wCOi0rV6eJboJ/Nr+CGdS+4Nw5Y4bqIqFuJ18qtHLgA3YEl1ezfCGgFrKv0/GygJ7ARr9vhImwQXmj2A5dY4LpjoeuYBjQfLzpSaxjZIrxwLbejwvfL8S64VbYHr+8WvGhYX2m7/cB3eC3biv3CtvZyKH6pAf3WdRHpzEYvJAANaLYE5Ra8a/PJrwjvIlrF0Qmf4fXJCt6FtXP85zcBc4BR/uuf+NsoXv9t6wrHmAIM9l/vCHwDPAf0jdHPkXoe04C+7LqIdGd9uglEgvIccIPrOkxK+hAYpQEtq3ZLE1PWvZBYbgQmuS7CpJxFwOUWuInBWroJRoLSAO+DdB/XtZiUsA3orwGtfEnSOGIt3QTj3wN/Dj++bm9MqAqBn1rgJhYL3QTkTwN5Nt5sBMaEoxi4SAM6w3Uh5ocsdBOUBnQJ3jX9fa5rMUmnFK8P19bmS0AWuglMAzoVb+CVDWQ3NVUGjNWAvuO6EFM1C90EpwH9D16LN7VunjCxoHiT2LzhuhBzcBa6SUADOhmvxWtdDeZQbtSAvui6CHNoNmQsiUhQBuMNcm/ouhaTUEqAcRrQl1wXYqpnoZtkJCgnAR8ALV3XYhLCXuBiDehHrgsxNWOhm4QkKB3w7lyzCQ3TWy4wUgM623UhpuasTzcJaUBXAwPw1k8w6Wk1cIoFbvKx0E1SGtDdwAjA+vHSz2y8wF3puhATOuteSAESlN8AD1L1ymMmtfwZuFUDmtrr66UwC90UIUE5C3iVqqcHN8lvH3CDBvQ114WYyFjophAJypHAm8AQ17WYqFoBXKABXeS6EBM569NNIRrQzcAw4F5+uGC5SV7/BPpa4KYOa+mmKAnKULzFzI90XIoJzx7gf+wOs9RjLd0UpQH9EjgRb8Fyk1w+Ao63wE1N1tJNAxKUn+Bd9e7guhZzSLuB22zxyNRmLd004E+Y0x34A7ZgeaL6EK91+7LrQkxsWUs3zUhQegJ/Afq5rsUAsAS4QwP6b9eFmPiwlm6a0YAuAE4Cfg6scVxOOtsOjAd6WOCmF2vppjEJSm3gBuB32Kxl8VIEPA08oAHd47oYE38WugYJSkPgDuB/gMMcl5OqivFGkjykAV3luhjjjoWu+S8JSmvgZrzWbzPH5aSKfXh96I9pQDe6Lsa4Z6FrfkSCUh+4CrgV6OS4nGS1C3gGeFoDut11MSZxWOiag5KgZADnAbcDAx2XkyyWAM8DL2lA810XYxKPha6pEQlKD2AscDnQ2nE5iWYf8C7wvAZ0uutiTGKz0DUhkaBkAj8BxgDnA/XcVuSMAtOBV4C3NaB5jusxScJC14TNH/VwAfBTYDjQwG1FMXcA+Bx4D/hAA5rjuB6ThCx0TVT4Y34HAWf5j25uK4qaHXi36L4PfKIB3eu4HpPkLHRNTEhQ2uKt4TYQOBno7LaiGlsJzAC+8h9LNGD/SUz0WOiauJCgNAf6A739Ry+gHe5uRS/Buw16GbAUmAXM0IBudVSPSRMWusYZv0uiPd6Uk+WPjv5zzYEmeHfIhbPgZj6QW+GxFViFF7LLgFUa0KKIfgBjwmChaxKaP1qiEV4AN/G/F6C0ikcJ3k0JubZarklUFrrGGBNHNrWjMcbEkYWuiToRyRSR+SLyoetajEk0FromFm7BGxFgjKnEQtdElYgcDYwEXnBdizGJyELXRNuTwF1AmetCjElEFromakTkHGCbqs51XYsxicpC10TTqcB5IrIWeAsYJiKvuy3JmMRi43RNTIjIUOAOVT3HdS3GJBJr6RpjTBxZS9cYY+LIWrrGGBNHFrrGGBNHFrrGGBNHFrrGGBNHFrrGGBNHFrrGGBNHFrrGGBNHFrrGGBNH/w9mi/JwsZ47/AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Figure 2 probability of corresponding numbers [0-9]:\n",
      " [ 0.13722247 -0.75562215  2.2758346   9.71087    -6.6896553  -0.92975795\n",
      " -7.2741666   1.2185297   2.0145504   0.16734621]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD5CAYAAACEcub7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dd3hUZdrH8e896Y0eSui9Sxs6MgoqKLoqVuyuvaxtfd3qxlF3V3fXtu+qq4u6a0NdVvfVtSJKgEWRiDTpCFIUCCW9J8/7x5lICAmZfmYm94drLsLMmTN3QvLLmec8537EGINSSqnwcNhdgFJKtSQaukopFUYaukopFUYaukopFUYaukopFUYaukopFUYauiqiiMgiEbk2RPvuISLFIhLn+XcnEVksIkUi8oiI/FJE5obgdS8VkY+CvV8VneLtLkAFl4jsAK41xnxsdy2RxhizE0ivd9f1wAGglQnShHUR6QVsBxKMMdWe130FeCUY+1fRT490VUvWE1gfrMBVyhsaujFMRK4Skf+KyGMiki8i34jIJM/9u0Rkv4hcWW/7WSLylYgUeh6/r8H+rhCRb0XkoIjcKyI7ROQUz2MOEfm5iGzzPP6GiLQ7Tm1ni8gqz2ttE5GZjWzTV0Q+8ezvgIi8IiJt6j3+MxHZ4xke2CQi0z33jxORXM++94nIo577e4mIEZF4Efk7cCVwj2fI4RQRuU9EXq63/ykisszztdslIld58XVa7Pk737PfiZ6v99J6+50kIitEpMDz96R6jy0SkQc8/29FIvKRiHRo+n9ZRRsN3dg3HlgDtAdeBV4DxgL9gMuAv4hI3VvuEuAKoA0wC7hJRM4BEJEhwFPApUAXoDXQtd7r3AacA7iALOAw8GRjBYnIOOBF4H88rzUV2NHYpsDvPfsbDHQH7vPsYyBwKzDWGJMBzKi3jyeAJ4wxrYC+wBsNd2yMuQrrLf8fjDHpDYdjRKQH8D7wv0AmMBJY1dzXyfO5ALTx7PezBvttB7wL/Bnr/+RR4F0RaV9vs0uAq4GOQCJwdyNfGxWlNHRj33ZjzAvGmBrgdazgut8YU2GM+QioxApgjDGLjDFrjTG1xpg1wDysEAU4H3jHGLPUGFMJ/Aao/7b8BuBXxpjdxpgKrHA8X0QaO29wDfC8MWaB57X2GGM2NtzIGLPVs02FMSYPK6Dq6qkBkoAhIpJgjNlhjNnmeawK6CciHYwxxcaYz/34ul0KfGyMmWeMqTLGHDTGrPLi69ScWcAWY8xLxphqY8w8YCNwVr1tXjDGbDbGlGH9whjpR/0qQmnoxr599T4uAzDGNLwvHUBExovIpyKSJyIFwI1A3VvbLGBX3ZOMMaXAwXr76Qm85Xkrng9swArGTo3U1B3Y1sj9RxGRjiLymmcIoRB4ua4eY8xW4A6scN/v2S7L89RrgAHARs/b9zObey1famzm69ScLODbBvd9y9HvGvbW+7iUo0/+qSinoavqexV4G+hujGkN/BXrLT7A90C3ug1FJAXr7XGdXcDpxpg29W7Jxpg9jbzOLqy3/c35PdbR9AmeoYLL6tWDMeZVY8wUrMA3wMOe+7cYY+ZgvT1/GJgvImlevJ63NR7v69TcSbnvPPXW1wNo7OukYpCGrqovAzhkjCn3jLteUu+x+cBZnpNAiYCbegGIFTy/FZGeACKSKSJnN/E6zwFXi8h0zwm4riIyqIl6irFOSnXFGgPGs/+BIjJNRJKAcqwj9hrPY5eJSKYxphbI9zylxqevhDXee4qIXOg58dZeROre5h/v65QH1AJ9mtjve8AAEbnEs9+LgCHAf3ysT0UpDV1V383A/SJShDVm+8MJKGPM18BPsE7EfQ8UAfuBCs8mT2Ad/X3kef7nWCfxjmGM+QLrRNFjQAGQw7FHf2AF+2jPNu8Cb9Z7LAl4CGue7V6so9pfeh6bCXwtIsWeui42xpR7+0Xw1LgTOAP4KXAI6yTaCM/Dx/s6lQK/Bf7rGWqZ0GC/B4EzPfs9CNwDnGmMOeBLfSp6iU5RVP7wzHjIB/obY7bbXY9S0UKPdJXXROQsEUn1jI/+CVhL41O9gvV6Mz3zb7eKyM+9fM7tIrJORL4WkTtCVZtS/tLQVb44G+tE0HdAf6y37SF5qyRWf4QngdOxxjzneOYKH+85w4DrgHFYQwFnikj/UNSnlL80dJXXjDHXemYltDbGTDfGbArhy40DthpjvvHMC34NK/SPZzDwuTGm1NP3IAc4N4Q1KuUzDV0VqbpSb14wsJuj57I2Zh0w1TPTIBXrRFj3ENWnlF+0y5iKVNLIfccdyjDGbBCRh4EFWFPNVgPVIahNKb/pka6KVLs5+ii1G9ZY8nEZY54zxow2xkzFmuq1JUT1KeUXDV0VqVYA/UWkt+dijIux5gEfl4h09PzdA5iN1RdBqYihwwsqIhljqkXkVuBDIA6rQc7XXjz1X56OXVXALcaYw6GsUylf6cURSikVRjq8oJRSYaTDC0qpiPTll192jI+PnwsMIzIPEGuBddXV1deOGTNmv7dP0tBVSkWk+Pj4uZ07dx6cmZl52OFwRNw4aG1treTl5Q3Zu3fvXOBH3j4vEn97KNXiiMidnn4R60Rknogk211TBBiWmZlZGImBC+BwOExmZmYB1pG4988LUT1KKS95egXfBjiNMcOwZmtcbG9VEcERqYFbx1OfTzmqwwsxRkS6Yy362BlrzOlZY8wT9lblH3FLItD+wMO0bl9GBpCC1Uc3ntlUch6JWM3Jyz23YqDwn38+/8AFn//Tp/65ESAeSBGRKiAVLy4EUaE3f/78VnfffXeP2tpaLrvssgO/+93v9jb/rOPT0I091cBPjTErRSQD+FJEFhhj1ttdWEPiljRgKNbbs+HAQKxm5O09twyAgiS+a19G1lFP7kAOTSwGuW9n52/d4s7EarJed9sJfO25rc022YdC8Cn5xRizR0T+hFVjGfCRZ9FQVZ/ImKDuz5gvj/dwdXU1d955Z48PP/xwc58+fapGjBgx+LzzzssfM2ZMQL/QNXRjjDHme6yVHTDGFInIBqxGMbaFrrglARiEFa51ATsM6EXjPRaOciiF0j75zW11RGlRaiuso8Ventsx3OLei9Ugp/7t62yTXez9KwWHiLTF6qDWG6sx/D9F5DJjzMvhrkUdsWjRorSePXtWDBkypBJg9uzZh+bPn99mzJgxAR3taujGMBHpBYwClof1da1hARfWcuPTsY5gE/zd38FUvD6yMAZTXpLc2otNO3tup9R/ulvc67CugvsQWJJtsisae3KQnQJs9ywzj4i8CUzCWv1Y2WTXrl2JXbt2raz7d7du3SqXL18e8MrMGroxyrOczr+AO4wxhSF/Pbd0wWqlOAs4lSAuG34wFV+Cr9AYhzeh2xjBOgofDtwNlLrFvQhPCGeb7FD1D94JTPC0oyzD+kWVG6LXUl5q7GpdEQn4xJ6GbgwSkQSswH3FGPNmc9v79RpuEWAsVsieiXVE3exQgT/yUqlsfitLbY2jEPA3dBuq68l7BoBb3DuwAvh94P1sk+11XcdjjFkuIvOBlVhj8l8BzwZj38p/PXr0qNyzZ09i3b93796dmJWVVRXofjV0Y4yICNYS5xuMMY8Gff9ucQI3AmdhnfQKuf1p3vfEramOLwlhKb2AGzy3A25xvwjMzTbZGwLdsTEmG8gOdD8qeFwuV8mOHTuSN27cmNirV6+qN998s90rr7zyTaD71dCNPZOBy4G1IrLKc98vjTHv+btDzxjthcCtNLGseigdSKXW220rKxJKQ1lLPR2Au4C73OJeBswFXs822eF6fRViCQkJPPLIIztnzpw5oKamhksuueSA0+kMeCqihm6MMcYsJUhv88UtXbGOaq8DOgVjn/44mOL9thVlSeE48dXQJM/tcbe452Ed/eqYbLA1M8UrFC666KKCiy66qCCY+9TQVccQt5wI/ARrUUfbv0cOpXq/bVlxSsBjbgFohWf4wS3uVcBjwCvZJrvGxppUhLH9B0pFBs8QwhVYQwgjbC7nKIeTifN229KiVK+HIkJsJPAP4F63uH8HvJRtsnW9NqW9F1o6cYuIWy4DNgF/I8ICF6AoyfuDg+KC9JDMoAhAP+B5YLNb3D92i9vrXyAqNmnotmDilhlY05ReookrtyJBcSKJzW/l2fZwRqR+T/fGmlWyzi3uc+0uRtlHhxdaIHHLcOBRjr4aK2KVxZPk7bZF+el+X/kWJoOAN93i/hz4ebbJzrG7IBVeGrotiLilDXA/cDN4P05qt4p4vJ6/UJyfES19aCcAi9zifgW4PdtkH7S7IBUekfpWTAWRZ9z2x8BmrFkJURO4ANUOvJ6/UJyf7sMEs4hwKbDeLe4L7S5EHW3r1q0J48ePH9CnT5+h/fr1G/rAAw8E5WIgPdKNceKWAVhn0SfYXYu/asT7Pg4lhWlpoawlRDoCr7vFPQe4Odtkf293QZFI3MFt7Wiyjz/v13NxxO4pU6aUHj582DFq1KghZ5xxRmGgrR31SDeGiVuuBL4kigPXI61GvLsqrbQoNVh9F+xwDtZR74/tLkRBz549q6ZMmVIK0LZt29q+ffuW7dy50+uTuk3R0I1B4pYMccvLwN8JYrcv2whSmESzfW6NobqiLDkjHCWFUBvgObe4P3KLu5fNtSiPTZs2Ja5fvz7V5XIF3G9ZQzfGeN6CrcQaK4wZh5ObD10MQb1c02anYk0vu8buQlq6goICx+zZs/s+9NBDu9q1axfwxTcaujHCc7LsLmAZ1oT8mHIohWYbydTUxBWFo5YwSgPmusX9lFvckT4VLiZVVFTIrFmz+l5wwQWHrrzySh/WL2mahm4MELdkAu8Cj4D3FxJEE29Wj6iuCmlbRzvdBCx0izssrTSVpba2losvvrjngAEDyu+77759wdqvhm6UE7dMA1YDp9tdSygd8CJ0qyoSy8JRi01OBHLd4g7u4oyqSQsWLEj/97//3X7p0qUZgwYNGjJo0KAhr7/+esAnanXKWBQTt9wE/IUW8MszL41mu4eVl9rS1jGcugNL3OK+Lttkv2J3MeHW3BSvYJsxY0axCUE7yZj/YY1V4pZ7gadoIf+HeanNrx5RVpzSEloopgAvu8X9iDbPiU4t4gc2lnhOmD2OdTlvi+HN6hElhWkBLxoYRe4C3neLO5rnJbdIGrpRRNwSjzX39nabSwm7g6k0G6jF+dE/JdlHpwIL3OJuY3chynsaulFC3JIMvInVaLzFOZjS/PdqcUFGS3y7PRb42C3utnYXoryjoRsFxC2tsJb+PsvuWuySn9z892rR4fSYnC7nhTFYU8ra2V2Iap6GboQTt3QEFgFTbS7FVt6sHlGcn+F1390YNAoreNvbXYg6Pg3dCCZu6QYswfqBatGKE2n2iqzignQflrCMSSOxgreD3YXEgtLSUhk+fPjggQMHDunXr9/QO++8MysY+9V5uhHKM6TwPjDA7loiQVk8zTYnLylMi/ZmN8EwAvjELe7p2SY7z+5igionN7gXhricx52Dm5ycbJYuXbqpdevWtRUVFTJ27NiBCxcuLJg+fXpAVz7qkW4E8sxS+CcwzO5aIoU3q0dEeVvHYBqOFbx6xBsAh8NB69atawEqKyulurpaRAJf91RDNzI9CZxmdxGRpLnVI4yhvLoyIdpWjQilYcD/ucXdkse5A1ZdXc2gQYOGdOrUaYTL5SqcNm1awP09NHQjjLjlf4Dr7a4j0jS3eoQxEkttHYNlEvCC3UVEs/j4eDZu3Lh+586da1auXJm2YsWKgNfg0zHdCCJuOQ942O46bLEF+ACoBZKBAqzGhrd4Hi8gfRpwwPNw4euM2f483D4DFn4NpRWm7Wg2s4QllFOOIFzLtSSSyDzmMYtZtKKVHZ+Z3ea4xb0l22Rn211INOvQoUPNlClTit55553WY8eO1eV6YoG4ZRzwEhD4oFG0qQXew2q7fgtQDsxqsM1HcG48ZWuAnwEdkqm4exZ8vhUeuhheui5x+0IWMtLzp5pqEklkE5voQpeWGrh17p0hH55rdxHR5rvvvos/cOBAHEBxcbEsWrSo1eDBgwMKXNAj3YggbukFvA3eLzUeU/YA7Tw3gNGe++rLg5GpFFBIymzgioO0OSUO4hxQVgmUx1c5cFBKKVvZSjrp1FDD53zOHOaE8ZOJLAZKP2DG6uVMeFmEScaw2u6aosWuXbsSrrrqqt41NTUYY+Tss88+NGfOnICHsTR0bSZuaY3VgLyT3bXYphCOOhBtBTSc7NQJXv+W+BOBt4DSauJKK2DSAHj0fdh/uKLnTGbyFm8RRxyzmMUKVjCCESTGZl/3ZtUi373A1YW76D7Rc9ebIjiN4bCthfmrmSlewTZ+/PiyDRs2rA/2fnV4wX6vAkPsLiLiNBxkOQ0+ryZxFJADZCZT6XBA2zRY9Gt495ruX6WTTle6cjVXk0suS1hCRzryNm/zOq+zi102fCL2KCN57aPcmbCL7oPq3d0HeEVEf+7tpF98G4lbbgXOsLsO27XCOtqtU4h1Eq3BNnf0Y/NXwG89dyXXu0atpCCt9hM+YRrTWM5yaqnlVE7lXd6lC104m7NZyMJQfhYR43s6L/0TPx1YTEZmIw+fDuhJNRtp6NpE3DIE+IPddUSELOAgcBioBtZhHZPVVwL7U63VI34PXNiH7+s//P66qjYZZNCe9hRSSBlldKc7VVQhnj/VzfdBj2oGar7AmfMMN0ypIf54Yyr3inBK2ApTRxFjWlLf58ggbkkEPkd7KhyxGWvKmAESgFKgBGt+2KnAbkhaQ1WPahKcwIJkKisg0eGA9CRIM60Kzs2/qXUKKbzESxRQgAMH05nOEpZQQQVppLGf/aSRxi2euWif8Akb2YggpJHGOZzT6EyHVaxiMYsBmMrUH2ZIzGMehRQylrGMYxwAb/M2YxlLF7qE/MtWx0DBfM7b+jXDvL1U9htgmDFE7Lpyq1ev3jFixIgDdtfRnNWrV3cYMWJEL2+31xNp9nATiYH7b6zwqz8/ts5/gQXA/3DsW//tWIFZ5wBwPjAY+BewD6uDRN2xVQ7WacP6o40DOH6XidFwZRbLnvkPLgCu4DNcno+Bt56evnnN0pSxAJdz+VFPHchAAHawg0QSeYu3fnhsEpOYxjQAPudzcsjhrAYdNEspZRGLuJ7rEYRneIaBDORbviWLLC7lUp7hGcYxjr3sxWDCGrjVxG1/huslj46+9CboA9yHNQNPhZEOL4RbTu74wik5P3JmDN5idynHGAlc1sj9BVjHRU11NuiNtUj4TcCVWEeqfYG9nsdvBnZizb8twpoONujY3TTnwHEuBC7OT2/2SqFe9CKlway85Hp9dOqGIhraxjb60pdUUkkhhb70ZStbiSOOKqqorbeS0Cd8wsmc7M2nExRFpOf+kbvb5dGxlx9Pv0uEEcGuSR2fhm445eQmAs9lxKcO+WL0P3q+OMi9KA5H5Cym2IvGZwp/gPUW3xvrgf5AIhAHVGFd/FCDNSPhU/A3kw6lNH3hSHFBmt9r9SxkIY/yKGtY02hgFlJ41JBDK1pRSCF96EMxxcxlLpOZzEY2kkVW2C7E2EafnEe5c3QFyf42+okH/qazGRq3evXqpLql1wcNGjQkPT191P33398x0P3q8EJ4/QIYCiAiiZd3PuOksztM/frEr65LWVOyteGpo8iwEWt2QWcvt18H1M0KzcQ6On4Gq+HgIawxWz/feR9v9YiSwnS/Q3e6588SlvAFX3h1pCoIccRxPucDUEMNL/ESc5jDB3xAAQWMYASD/Dmkb4aBik+YtmIJJ7qa37pZY4GfAE8EYV8hJUJQWzsaw3Hn/Y4YMaJi48aN68FqfNO5c+cRF198cX6gr6u/4cIlJ3co8MuGd7eKTx+6yvlq1l8H/CLHgTS74m1YVWK1UPf2yLQI2A/0q3ff6VjDDpM4cpS7GHgDjv8tf6zC46weUVacEvDijMMZznqOnQtfd2T7Qx0UksHRrXtXsIKRjGQ3u4kjjgu44IcTb8FUi+x/kcu3LOHEKUHc7YMi9Aji/mLO22+/3apHjx4VAwYMqAx0Xxq64ZCT6wDmQuOXRolI8g1Zs115kz9eNzCl57fhLe44DntuTwOPYc2ffQYrXBvzNdZYbWPLQ27EmhpWhRXMFwKrsYLdSyVNXFpmDMW1NXHNrizRmIMc/OHjTWyiA8e2oO1LX7axjTLPn7ox3jpllLGZzYxgxFHjwsGeolZB4vrHub1mO32C3Wc5HaudqF9EZKCIrKp3KxSRO4JYn+3mzZvX7vzzzz/Y/JbN0+GF8LgMmNDcRu0SWp2wYdw/Sx7b/eriu7c9fqKxu/lNJ+Ceev9+DKvpZMPZC3XWQqOzP2uwJshdgjXEUMd4HvNSWTyN9oY1tVIIx2/9CDCf+exgB6WU8giPcDIns4UtHOAAgtCGNpzJmQDsYQ+55HI2Z5NKKlOZyrM8C4ALF6n12vvmkMNUpiIIfenLF3zB0zyNE6f3n1wz8uiw7BmuH1VNyHoGnynChcbwhq9PNMZswjoNi4jEYZ0qfeu4T4oi5eXl8vHHH7d+9NFHdwdjfxq6oZaTGw/8xtvNRSTtru6XTr2s0xlfTVx5deY35Xu6hbC6o80HdmDNkX0EayhgdBPb7gFygbM9/z6MdSTcs5Ftv8D6kUzkSIeJp7BOuPkQIU2tHlFdHV/szfPrxl/rG93EJ9jV86f+dk1tO5OZP3ycQAJXcIU35XjFgFnDCYvf4txgjN825wkRPjKGQMYtpwPbjDGR844tQPPnz289ZMiQ0u7duwflrYuGbuhdBfXei3qpY2LbUVvHv1X0u50vLPn19qdPDH5ZjTg2k452Z72Pu3puddoCP23ieRPrfSxevE4Tmlo9oroyodS/PUY2A0Vvc9aGrxgdjsAF63Tp77FG4f11MTAvOOVEhtdee63dhRdeeKj5Lb2jY7qhZE0R+7W/TxeRjF/1/PGJeya+l9s9qdP3zT8jttUIjS48WVGWGHCP00hTg2Pns1y39ytGjwvzS18jQm9/nigiicCPsNb3iwlFRUWOpUuXtrrssssCnrVQR490Q+saGn/D7ZOspEznjgnvFNy7/emlv9v5QjDPWkcXIa1GqI0zRx8slJekxFRThRJSv3qSm3uWkmbHjIIErFk21/nx3NOBlcaYfcEtydLcFK9QyMjIqM3Pz18VzH3qkW6o5OQm0cgUMX85RFr/ts/NU3ZMePuLLontY2tpbR8UJnHM+G1JUWrkXGASoJ10X/wIdw0vJa1d81uHzJUifh0szCHGhhZCQUM3dK4Hgn4SrGdyl3G7J74Xd0e3OcuCve9ocDj52AlrJYVpUd+1yUDVUiYtfp4fT60lzu53oAnAr3x5goikYl23+GZIKoohGrqhkJObgnX1WUg4xNHusX53Tdo87l+fZSa0DcrcwWhxMJVjTpoV56c3NjM4ahg4OI+Lv/6YU6faXUs9V/lytGuMKTXGtDfG6KrMzdDQDY2b8PtiV+/1T+0x8ftJH9Re1+Xc5aF+rUhxMIVjTpoVHc6I2tCtIn7L//KTss0MHGl3LQ0k0PR8FBUADd1gy8lNI4zt8uLEkfnswF+OXzf2tWXt4lsF7QxrpDqQSkXD+4rzM5rtMBaJDtNm+R+5u8sh2oVvLrZvrhYh4Mur1dE0dIPvFiDgTkS+GprWd9K+yR+VX97pjBXhfu1wOpBmrR5RX5EXbR0jzQYGLXqC28ZVkuR3o54wSAdusLuIWKOhG3y2fZPGS1znFwe7x3455qUlreLSCpt/RvTJSz22oUFJQXpTFyZHHAOl7zPjs9e56CQQey/z9s5PRPCrr0UscLvdHfv16ze0f//+Q88666zepaWlAf+f2X2WNLbk5E7m2NW9wm50xqATD0z+eM8lG369dX7ewqYu5I1KeWkc04mttCjV336yYWUtiX5VwS56TGx+64jRFbgIeNnuQtziDmprx2yTfdx5v9u3b0949tlnO23atGldenq6OeOMM/rMnTu33W233RbQyWs90g2uy5vfJDwSHPFd3xjy+1H/HfXc4jRHSond9QTLwQbdF4zBlJckh6dreACOLIneY7DdtfjhdrsLsEtNTY2UlJQ4qqqqKCsrc3Tr1u2Y4S1faegGi3UxxIV2l1GfiMik1idMPTRl4cFZ7aestrueYGhk9YhCYxwR/X3czJLo0cAp4nv/kGjXu3fvqltuuWVv7969T+jYseOIjIyMmtmzZwc8bBfR36xRZhZW25eIk+hI6PHOsEdPWDjiqZwUR1LErv7qjYarR9TWOCJ27NqHJdGjgZ9tiqJXXl5e3Lvvvttm69ata/fu3bumtLTU8dRTTwV8paCGbvBEzNBCY0REprUd6zo85ZO909uMXWd3Pf5quHqEt20dw82zJPqq95gVrg5hodbiQvedd95p1aNHj4qsrKzqpKQkc8455+QvW7Ys4NkmGrrBkJPbDjjD7jK8keRI7L1gxJOD/zP8sUVJknDMnNdI13D1iKqKhIg7cq8mbvtT3HToa4YF9cSPzZwi9LK7iHDq1atX5cqVK9OLiooctbW1fPLJJxmDBw8OuKOdhm5wXEQTS/FEIhGJm9V+ykmHp3y6a3KrERvsrscXDVePqChLiqhfHPWWRPerPWKEa1FHu9OmTSs566yzDp9wwgmDBw4cOLS2tlbuuuuugJtN6ZSx4IjooYWmpMQl9Vsy6m/V/zrwyaJL1v96cpWpjvj5mA1XjygrTgn4bHKwbKNPzstceqIhsk/sBeB84E92vXhzU7xC4bHHHvvuscce+y6Y+4zVb47wycntx9FrI0QVEYk/P3P6SYemLNzmzBi8xe56mlPlOHqFttKiVNtXULaWRD956Utc7jo2cOuWD6u7tQIeb7CHAuAsrHXqhwIv1HvuGM/9n3nuq8ZaiM6WxTLGidDdjheOJRq6gbvM7gKCIT0uddAXo//R88VB7kVxOCK2P22tHL0AZXFBuq1XddUtib6YqU00lx8IrPLcvgRSgXMbbPMkMARreeRFWH1mKrGWXn4Ia/G6ugPMp7HeWDW6clGoCXCeHS8cSzR0A3d285tEBxFJvLzzGScdmrJw4wlp/b6xu55GeVaPqPtn8UDge/EAABVzSURBVOEM276HK0jc8AS3+bAk+kKs5fIadkwUrHXtDVAMtMMa+UsAyrCOahOAfOAdCOLCl364wM4XjwUauoHIyc0AhttdRrC1ik8fusr5atYzA36Z40Bsf/veUGHSkUbmRfnptoxDH6D9sj9yd68C2vjQwvM1rMUVGroV2ABkYX07PYH1o3kL8ChwI9YiJPdj9Ra39eB+oghZdhYQ7TR0AzMBiNperscjIsnXZ53rOjD543WDUntF1HLah1KOLNlTdDgj6XjbBpsBs5oTFv2FWydVk+DDAvKVwNs0fqD4IdZ473dYwxC3Yq1n3wNruOEzrOGE74BBWMMLFwGb/f00AqFDDAHS0A3MJLsLCLW2Ca1OWD/2jczH+92VI9b7X9sdSjlyFqmkID1sg5ueJdFXvMW5J/n+7PeB0UCnRh57AZiNlWf9gN7Axgbb/Ap4APgzcCng9txsMcOuF44FGrqBmWx3AeEgIqm3d5vj2jfpo1V9U7rttrue+qtHlBSmhaUfbeBLos+j8aEFsI5oF3o+3oc1a6F+s7ocrGZf/bHGdx1Yb7BsmcEAMN6uFw63Bx54oGP//v2H9uvXb+j9998flD7ZOk/XXzm5cVjDCy1GZmLbUVvGvVn00M6/L/nl9qdOtKuO+qtHlBSmhbzDWOBLopcCC7BmI9T5q+fvG4F7gauwxnMN8DDQwfO4AR4E3vD8+3qsI91qrJkMtuggQl9j2BbWV31VgnuF3yXmuPN+V6xYkfziiy9mrly5ckNycnKty+UacO655xYMHz48oAtyNHT9NxzIsLuIcBORjF/0vPrEqzqflTtx5dVdv63YG/K14BrKS6MSwBiqK8uTQvp/sItui1/gqkmBrdCbCjRswXpjvY+zgI+aeK5gBXadwcBK/0sJngkQ5tANs7Vr16aMHj26OCMjoxZg8uTJRa+//nqb4cOH7wtkvzq84L8WMbTQlC5JHZzbJ7yd+queP14a7tfOS8OaR2wI2cqzdUuiP8c1kbAkeiSK+Xd5I0eOLFu+fHnG3r1744qKihwLFixovWvXroAv99fQ9V+LDl0AEWn9YO+bpnw74Z0vshI77A/X6x5Itebp1tTEhaSto4GDr3FRpC2JHmHMWLsrCLXRo0eX33777XunTZs24OSTT+4/ZMiQ0vj4wH//auj6L+ZnLnirR3LncbsmvptwV7dLl4Xj9epWj6iuig/6mSRrSfRbSzcxKNKWRLeRqUhPqdkwdlDJkp9d8v2SRU9sXF/20crB5OTG/DuAO++888D69es35ObmbmrXrl1N//79A+4yFvNftJDIye3KsZcVtWgOcbR9pN8dk27qet5nk1de239/1aEOzT/LP4c8oVtVkRjUto6HabP8aW4cGuEr9IaYKWuVVvPNsN7lB6ePLnTMHFeQ6RxU2icx3jS2zFB/rKs6YtaePXviu3btWr1ly5bEd999t80XX3zRcC6fzzR0/aNHuU3ol9J94neT3s+7ZfPDnz/z/ZshGfc7nEwchpry0uC1ddzAwJzXuWhqlKzQGySmpG16zTfD+5QdPtVZ6JgxrqDTyH6lvRPiGerlDoYT46H7ox/9qG9+fn58fHy8efzxx3dmZmYG3JdEQ9c/Q+wuIJLFiSPzrwN/kfmTbhf9d+pX1w09VF3YJpj7L0oinnJqyopTA/4BMFD6ATNWL2dCrKzw0ART2L5VzTcj+5UWnDa2MP60sQVdhvcp6xXnCOgy9uEcmcsWes1M8QqFL7/8clOw96mh659edhcQDYam9Zm8b/JHe3+88f4VL+17L2gnXkoSSaScipKCtID6QkTpkujNEkx+Zpvqb0YPKC061VmYOGNcQZfBPct7OoRgj1PHXN+RcNDQ9Y+O53opXuI6vzjY3fmObnOWnLzqxhGFNSUBX8xQmkASUFRckOb3UEA5SWv/wi2di8mIxiXRfyBiDnZqW7XDObC0aMa4guTTxhZ2HdCtojvWNcehpqHrBw1d//Syu4BoMzpj0IkHp3z83SXr793yz7yPA7qyqDLOaiZbnJ/h1/fvXjot/RvXjou2FXodYvZ3aV/17bjBJSUzxxWknuos7Na7S2UW0N6mkrShuR80dH1lXf7bze4yolG8xGe9MfT3WZ8VXLL4tDW3jC6uKfNrlkCVwwrdovwMn9o6GqjJZczSdzkz4sdv4xzm+66ZlbsmDCkpmzmuIOWUMYU9u3es6gQE5fr/IEkgJzcDl7Oo+U39UltbWysOhyMiGi01pra2VgCfhrk0dH2XhdVRWvlpYuvhUw9N/mTXeV/fs/Wdg0t8HmesWz2iOD/d67aOBgr+xewt6xgecYEbH2f2dO9YuWvSsOKK08cXpE0fXdizc7vqLkDYL7H2QzsgVKG7Li8vb0hmZmZBJAZvbW2t5OXltQbW+fI8DV3fRcMPQsRLcMR3/79hj3RblP9lzqy1d4wrq63wvjetZ/WI4oL0tOY3tpZEf4bryaOj0++Cg8KYhHizq1fnyj1ThhdXnj6+IOPkUYW9O7Su6YrVRiwatQdC0m+5urr62r17987du3fvMCLzQq5aYF11dfW1vjxJQ9d3mXYXECtERE5u63QdnvLJ9jPX3ln88eEvvD4xU5pEbUlB820di0jPfZKb+5eT0jqwan1lTGKC2dGnS8X3U0cUV50+rqD1SaOKerdJr+mB1csxVrQL1Y7HjBmzH/hRqPZvFw1d32noBlmSI7H3Ryf8pfaDQ8sWnbvufyZWmKpmhw1Kk6gpLU49bpB6lkSfYnCEeHUPU5OcaLb361q+76SRRdUzxxe2PXF4Ue9WabW9sTqSx7KQhW6s0tD1nYZuCIiI4/T2k086POXTraetubVqacGq407lyieuqrqy8eVyDFR8yskrFjM1BOO3pjo1qXbbgO4VeSePKqw9fXxhu0nDivukJdf2w1r2oaWxa+ZE1NLQ9Z2GbgilxCX1Wzzy2eq3Dny66OL1v5pcZaobPWm5vyK50avRapH9L3Ppvm/o28SS6L4wlekptd8M6lGeN210IaePL2g/YUhJ3+REMxBrbXWlR7o+09D1nYZuiIlI/OzMaScdnvLJppNX3SgritYPaLjN3vJjLwGuIHHDU9zUpoA2fkzaN+WtUmu3De1ddnD66ELHjHGFHcYOKumTlGAGYa0GqRqnoesjDV3f6XSxMEmLSxm4fPTfq+bt/3DRlRvvm1Jtan74ft1bmn7U1WgHaL/sr9wwyrsVek1JG6vRy6FTxxTGzxhX0HFUf58avagjdHjBRxq6vquyu4CWREQSLuk086Sz2p+4fuqq65NWFW/uC7C3OMORiLUk+hqGL36L2U2M35qidq1qvhnRtzT/tLGF8TPGFnQ+oW9Z7wAbvagj9EjXRxq6vtPQtUFGfNqQlWNernhu7/8tumHT76YeKMyI7wLFb3PW+q8Y7QIQTEGHNtXfjOpfWniaszBhxriCLkN6lfdyCCPsrj+G6ZGujzR0fVdpdwEtlYgkXdvlnJPO6zB97QNfvFWxOGvOZ1k9OiZcPW7n56eNLcga2L2iBzDK7jpbmIi7UizSaej6To90bdY2IWP4o/dcMRwKsW7KRqG6BDhmReKldZFOQ1epIzR0faSh6zsNXaWO0ND1kYau7zR0lTpCQ9dHGrq+09BV6ggNXR9p6PpOQ1epIzR0faSh6zudMqbUETp9xEcaur7TI12ljtAjXR9p6Ppun90FKBVBNHR9pKHruy12F6BUBNHQ9ZGGru+22l2AUhFEQ9dHGrq+cjlLgO/tLkOpCBGSRSljmYauf3SIQSnYh8uZZ3cR0UZD1z8aukrBWrsLiEYauv7R0FVKQ9cvGrr+0dBVSkPXLxq6/tEZDEpp6PpFQ9c/W9GO+aplqwXW211ENNLQ9YfLWQp8Z3cZStnoG8/PgfKRhq7/NttdgFI20qEFP2no+u8zuwtQykYaun7S0PXfR3YXoJSNNHT9pKHrv8+AYruLUMomGrp+0tD1l8tZCeTYXYZSNvgel3OT3UVEKw3dwOgQg2qJPrS7gGimoRuYBXYXoJQN3re7gGimoRsIl3MDsMvuMpQKoxr0YCMgGrqB029A1ZIsx+U8bHcR0UxDN3Aauqol0aGFAGnoBm4B1nXoSrUEb9pdQLTT0A2Uy3kQ+MruMpQKgw24nNrkJkAausHxnt0FKBUG8+0uIBZo6AbHi2irRxX7NHSDQEM3GFzOrcASu8tQKoQ243KusbuIWKChGzzP2V2AUiE0z+4CYoWGbvDMBwrtLkKpEKgAnra7iFihoRssVhf91+wuQ6kQeBWXc5/dRcQKDd3g+qvdBSgVAo/aXUAs0dANJpfzK/SEmootC3A519ldRCzR0A2+J+wuQKkgesTuAmKNhm7w/Rv41u4ilAqCr3E5tXdukGnoBpvLWQM8ZXcZSgXBY3YXEIs0dEPjb+j6aSq67QdetruIWKShGwpWv9E/2F2GUgF4Epezwu4iYpGGbuj8CdhtdxFK+aEcHSILGQ3dUHE5y4Bf2l2GUn74Gy7nAbuLiFUauqH1MvCl3UUo5YM8INvuImKZhm4ouZwGuMvuMpTywc90DbTQ0tANNZdzMfCW3WUo5YVlwN/tLiLWaeiGxz1Ald1FKHUcNcDNnndnKoQ0dMPBanL+pN1lKHUcT+Fyrra7iJZAQzd87gcO2V2EUo3YB9xrdxEthYZuuFgnJ+63uwylGnEPLmeB3UW0FGKMDuGETU5uAvBfYKzdpYRSflER1/7xQdZt34aI8PzP7uXNxZ/yzrIlJCYk0DerGy/87De0ycjw6rkTh57Az575X95fvoyR/Qbw4i/dALz00XscKizg9vPnhPtTjCVLcDmn2l1ES6JHuuHkclYBc4jxZX1u/8sjzBw3kY0vzWf1c68yuEdvTnWOZ90Lr7Hm+XkM6N6D37/6d6+fW1BczLJ1a1jz/DxqamtY+81WyirK+fsH73DzOReE95OLLdXALXYX0dJo6Iaby7kNuMHuMkKlsKSYxau/4ppZZwOQmJBAm4wMThs7gfj4eAAmDBnG7rxjV39p6rkOh1BZXYUxhrKKChLi4vnjay9x2+yLSfDsU/nlz7ica+0uoqXR0LWDy/kaMbp68Dff7SGzTRuufsjNqGsv5do/PEhJWdlR2zz/3tucPm6S18/NSE3jvKnTGHXtpfTukkXr9HRWbFzP2VNc4fq0YtFy4Bd2F9ESaeja5zZgvd1FBFt1TQ0rN2/iprPP56u5r5CWksxD9YYSfvvS88THxXPpqaf79Nx75lzBqude5ZGb7+Te5/7K/T++kbn/+TcX3vcLHnwxJn9/hdJ+4Hxczkq7C2mJNHTtYq0efBFWR6eY0S2zI90yOzJ+yDAAzndNZ+WWTQD844P/8J/PlvLKrx9ARHx6bp2vPP8e0K0HL370Hm/c93vWbd/Glt07Q/lpxZIa4GJcTu2AZxMNXTtZC/7daXcZwdS5fQe6d+zEpp07AFj45QqG9OzNB8uX8fC8F3n7d4+Qmpzs03Prs45yb6Cqupqa2hoAHA4HpeUx9bsrlH6By/mp3UW0ZDplLBLk5P4TON/uMoJl1ZZNXPvH31JZXUWfLl154ee/YewNV1JRVUn7Vq0BmDBkOH/96S/47kAe1/7xQd57+Ikmn9s2oxUA/16yiNXbtpB91XUA3P3U43y44nNO6NuPV379oD2fbHSZj8up0z1spqEbCXJyWwOrgF42V6Ji1wZgHC6nLiNlMx1eiATW1UAXo01xVGgUAbM1cCODhm6kcDmXA5cDtXaXomLO1bicG+0uQlk0dCOJy/k6MXzhhLLFH3A5/2V3EeoIDd1I43LORVebUMHxFrpOX8TR0I1ELudjgNvuMlRUew9rPm6N3YWoo+nshUiWk/soMTaPV4XFQuBMXE6dvByBNHQjXU7uXOAau8tQUWMJMNNzxaOKQDq8EPmuB96wuwgVFT4HZmngRjYN3UjnctYClwHv2l2KimifAqfichbZXYg6Pg3daGA1Pz8f+NjuUlREeg84Qy9+iA4autHCOilyBvCi3aWoiPIv4Fw9aRY99ERaNMrJ/Q06pUzBP4BrdFpYdNHQjVY5uZcCzwOJdpeiwq4KuBuX8892F6J8p6EbzXJyp2K9vexgdykqbPYAF+JyLrO7EOUfHdONZi7nYmAMsNLuUlRYfAqM1sCNbhq60c7l3AlMxhrfU7HrD1hTwvbbXYgKjA4vxJKc3FuAx4AEu0tRQVMIXInL+W+7C1HBoaEba3JyJwJ/BwbYXIkK3FrgPFzOLXYXooJHhxdijcv5GXAC8Ft0JYpo9jIwQQM39uiRbizLyR0O/A0Yb3cpymt7gJ96GtqrGKRHurHM5VwLTAJuB/QS0chWBTwMDNTAjW16pNtS5OT2AJ4CZtldijrGAuAnuJyb7C5EhZ6GbkuTk3sx8ATQ0e5SFN8Cd+Fyvml3ISp8dHihpXE5XwMGAy8A+hvXHhXAg8BgDdyWR490W7Kc3MHAz4FLgHibq2kp3gVux+XcZnchyh4augpycnsB9wBXA8n2FhOzFgJ/xOX80O5ClL00dNURObmdsZZ/vwlIt7maWFAOvAo87plJopSGrmpETm5b4DbPrZ3N1USjfVgzRZ7G5cyzuxgVWTR0VdNyctOBG4CfAl1sriYarAYeB+bhclbYXYyKTBq6qnk5uQnAacAFwNlAG3sLiii1wH+whhA+tbsYFfk0dJVvcnITOTqAW9tbkC0MsByrgfx8XM4d9pajoomGrvKfFcAzOBLArewtKKRqgMXAm8BbuJx7bK5HRSkNXRUcOblJHAngM4iNE3C7gQ+BD4CFuJyHba5HxQANXRV8ObkCDAQmem6TgCGA2FlWM2qArcDXwFLgQ1zO9faWpGKRhq4Kj5zc1sBoYARWv98RWEEc7osxDLAdK1zX1ft7o844UOGgoavsk5Mbh7XCRW+sFY07AO3rfVz/3+1p+lLlCqAEq31l/VvdfXs4Eq4bcDlLQvMJKdU8DV0VPayj5Q5Ya8AdCVeXs9rWupTygYauUkqFkbZ2VEqpMNLQVUElIski8oWIrBaRr0XEbXdNSkUSHV5QQSUiAqQZY4pFJAFr+tXtxpjPbS5NqYigjatVUBnrt3jdIpgJnpv+ZlfKQ4cXVNCJSJyIrAL2AwuMMcvtrkmpSKGhq4LOGFNjjBkJdAPGicgwu2tSKlJo6KqQMcbkA4uAmTaXolTE0NBVQSUimSLSxvNxCnAKsNHeqpSKHHoiTQVbF+AfIhKH9Uv9DWPMf2yuSamIoVPGlFIqjHR4QSmlwkhDVymlwkhDVymlwkhDVymlwkhDVymlwkhDVymlwkhDVymlwkhDVymlwuj/AeICFSuaP6t/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# define the pie drawing function of probability analysis\n",
    "def plot_pie(prbs):\n",
    "    dict1 = {}\n",
    "    # remove the negative number and build the dictionary dict1. The key is the number and the value is the probability value\n",
    "    for i in range(10):\n",
    "        if prbs[i] > 0:\n",
    "            dict1[str(i)] = prbs[i]\n",
    "\n",
    "    label_list = dict1.keys()\n",
    "    size = dict1.values()\n",
    "    colors = [\"red\", \"green\", \"pink\", \"blue\", \"purple\", \"orange\", \"gray\"] \n",
    "    color = colors[: len(size)]\n",
    "    plt.pie(size, colors=color, labels=label_list, labeldistance=1.1, autopct=\"%1.1f%%\", shadow=False, startangle=90, pctdistance=0.6)\n",
    "    plt.axis(\"equal\")\n",
    "    plt.legend()\n",
    "    plt.title(\"Image classification\")\n",
    "    plt.show()\n",
    "    \n",
    "    \n",
    "for i in range(2):\n",
    "    print(\"Figure {} probability of corresponding numbers [0-9]:\\n\".format(i+1), prb[i])\n",
    "    plot_pie(prb[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上过程就是这次手写数字分类训练的全部体验过程。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
